-- Logs begin at Mon 2022-08-01 21:41:53 UTC, end at Mon 2022-08-01 23:05:05 UTC. -- Aug 01 22:28:29.191964 dv-cinder-851640-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 22:28:29.204527 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 22:28:29 2022] *** Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: nodename: dv-cinder-851640-1-iscsi Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: machine: x86_64 Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: clock source: unix Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: pcre jit disabled Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: detected number of CPU cores: 4 Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: current working directory: / Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: detected binary path: /usr/bin/uwsgi-core Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: your processes number limit is 47680 Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: your memory page size is 4096 bytes Aug 01 22:28:29.206647 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: detected max file descriptor number: 2048 Aug 01 22:28:29.207325 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: lock engine: pthread robust mutexes Aug 01 22:28:29.207325 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: thunder lock: enabled Aug 01 22:28:29.207325 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 22:28:29.207325 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 22:28:29.245939 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: Python main interpreter initialized at 0x5638a7d769c0 Aug 01 22:28:29.245939 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: python threads support enabled Aug 01 22:28:29.245939 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: your server socket listen backlog is limited to 100 connections Aug 01 22:28:29.245939 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: your mercy for graceful operations on workers is 90 seconds Aug 01 22:28:29.246312 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 22:28:29.246312 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: *** Operational MODE: preforking *** Aug 01 22:28:29.246625 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: *** uWSGI is running in multiple interpreter mode *** Aug 01 22:28:29.246625 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: spawned uWSGI master process (pid: 80388) Aug 01 22:28:29.246786 dv-cinder-851640-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 22:28:29.249606 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: spawned uWSGI worker 1 (pid: 80398, cores: 1) Aug 01 22:28:29.249606 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: spawned uWSGI worker 2 (pid: 80399, cores: 1) Aug 01 22:28:29.249606 dv-cinder-851640-1-iscsi devstack@keystone.service[80388]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 22:28:34.712327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 22:28:34.712327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Aug 01 22:28:34.713170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 22:28:34.713170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Aug 01 22:28:34.715235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 22:28:34.715235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Aug 01 22:28:34.715753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 22:28:34.715753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Aug 01 22:28:34.762067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 22:28:34.762256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 22:28:34.762416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] command line args: [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 22:28:34.762685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 22:28:34.762880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ================================================================================ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 22:28:34.763125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.763254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 22:28:34.763344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 22:28:34.763405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_dir = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.763546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] command line args: [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 22:28:34.763676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.763676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_source = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.763814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 22:28:34.763897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] debug = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.763976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ================================================================================ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 22:28:34.764052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_dir = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_source = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.764957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] debug = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_config_append = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_dir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.765960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_config_append = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_dir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.766946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.767962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.768945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] publish_errors = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.769984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.770899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] publish_errors = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] standard_threads = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.771933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_journal = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_json = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_stderr = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] standard_threads = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_syslog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.772888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.772989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.773112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.773198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_journal = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.773326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_json = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.773408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.773484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_stderr = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.773602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.773680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_syslog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.773849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.773927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 22:28:34.774081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.774871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.775989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.776948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.777979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.778953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.779881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.780874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.781973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.782866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.783996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.784990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.785920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.786941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.787940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.788885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.789977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.790998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.791936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.792892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.793972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.794945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.795879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.796999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.external = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.797977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.password = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.token = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.external = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.798969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.password = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.token = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.799920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.800851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.801976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.802928 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.803935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.804959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.805956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.806883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.807977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.808981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.809925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.810874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.811885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.812949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813506 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.813991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.814931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.815992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816071 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.816929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.817959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.818884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.819980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.820903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.821968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.822976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.823904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.824991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.825942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.826948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.827895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.828900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.829910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.830925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.831874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.832965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.833955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.834912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.835986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.836986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.837976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.838885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.839998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.840942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.841998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.842979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.843140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.843564 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.843732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.843810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.843963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844506 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.844969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.845835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.846942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 22:28:34.847648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.847894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.848130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.848380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.848616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.848862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.849093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.849335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.849591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 22:28:34.849773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 22:28:34.876615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.876841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.877064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.878078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.878315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.878545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.885330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.887307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.915791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.915968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.919448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.919689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.967274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:34.968622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.017263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.017498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.018676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.018917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.082963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.083906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.084024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.084106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.084248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.084248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.107989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.108878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.109088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.119480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.120018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.132211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.132509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.132654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.132864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.132988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.133934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.134123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.134366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 22:28:35.150567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.150753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 22:28:35.151114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.151301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.151398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 22:28:35.151543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 22:28:35.151671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.151792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.151911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.152031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 22:28:35.152158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.152330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.152452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.152580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.152779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.153032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.161956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.162224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.162490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.162768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.162855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.163101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.163369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.163652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.169587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.170585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.175583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.175866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.176128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.176392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.176697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.176956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.177223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.177496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.177785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.178073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.178308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.179013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.179317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.179583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.179861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.180138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.180398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.180660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.180926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.181189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.181452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.181745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.201326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.202955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.203239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.204793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.206308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.206583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.207865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.208148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.208407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.211186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.211462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.211738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.215098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.215385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.218749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.218968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.219898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.220189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.220474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.220757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.221029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.221316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.221593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.221903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.223065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.223346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.223620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.223884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.224152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.224432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.224707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.224977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.238197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.238469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.240812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.241089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.242405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.244997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.245588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.245879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.248149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.248418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.250303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.252857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.253489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.256020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.256741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.257039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.257332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.257601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.257908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.258176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.259186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.259458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.259772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.260054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.260339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.260609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.274392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.274673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.274939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.275201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.276602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.276879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.277147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.277376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.282192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.283743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.284028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.285354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.286874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.287017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.287134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.287225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.287445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.287738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.288006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.289961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.290250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.290550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.290828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.291110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.299569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.299851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.300130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.300408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.300691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.300965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.301239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.302522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.302770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.303068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.303344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.303615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.303861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.304126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.316832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.319556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.319830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.322506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.322840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.325330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.325456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.327345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.327630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.327920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.328046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.329503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.329807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.330080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.334436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.336041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.336484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.338100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.338995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.339282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.339570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.339866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.341000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.341281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.341580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.341860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.348320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.348607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.348894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.350219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.350502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.350779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.355749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.356031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.356312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.356589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.356877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.357141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.357425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.357516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 22:28:35.357712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.357820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.358958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.359860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.360145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.360427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.360686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.360963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.361223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 22:28:35.393136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5638a7d769c0 pid: 80399 (default app) Aug 01 22:28:35.394605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5638a7d769c0 pid: 80398 (default app) Aug 01 22:28:35.396515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e857ce-4944-43e4-ba43-ea885a3479eb None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:35.396805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e857ce-4944-43e4-ba43-ea885a3479eb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:35.397083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e857ce-4944-43e4-ba43-ea885a3479eb None None] PATH_INFO: `/v3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:35.401440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Mon Aug 1 22:28:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:43.088441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:43.089010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:43.089010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:43.093971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:28:43.103169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:43.103925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:43.104069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:43.184590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: INFO dbcounter [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] Registered counter for database keystone Aug 01 22:28:43.205983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG oslo_db.sqlalchemy.engines [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 22:28:43.218221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writer thread running {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 22:28:43.406822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 22:28:43.407158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 22:28:43.461565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:43.463205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:43.599921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 22:28:43.757003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:43.833859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: INFO dbcounter [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] Registered counter for database keystone Aug 01 22:28:43.860258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG oslo_db.sqlalchemy.engines [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80398) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 22:28:43.872513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writer thread running {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 22:28:43.991300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453026f3-8206-4689-823a-b3371e3eda39 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:43.996817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:43.997463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:43.997784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:43.998055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:44.007354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:44.058520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 22:28:44.058850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b82d5-f0a8-48fa-a9d3-304d4e110d35 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:44.069585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Mon Aug 1 22:28:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:28:44.076751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:44.093817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe2892-98bf-4aac-aff3-9b456ab7598f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:44.099625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:44.100765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:44.101053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:44.101333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] PATH_INFO: `/v3/projects/e92bd71b1dab4dce929247e99aba6067` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:44.102948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] RBAC: Authorizing `identity:get_project(project_id=e92bd71b1dab4dce929247e99aba6067)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:44.153204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 22:28:44.153827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e7fde-ff0f-4da5-a0bd-cba5dda7adf4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:44.158191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Mon Aug 1 22:28:44 2022] GET /identity/v3/projects/e92bd71b1dab4dce929247e99aba6067 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:46.157081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcdf58-ae48-47fe-a319-bc7f78a7a7fa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:46.157795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcdf58-ae48-47fe-a319-bc7f78a7a7fa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:46.157795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcdf58-ae48-47fe-a319-bc7f78a7a7fa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:46.161850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:46.170695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:46.171419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:46.171547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:46.205259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:46.207636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:46.221021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 22:28:46.243007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcdf58-ae48-47fe-a319-bc7f78a7a7fa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:46.259121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcdf58-ae48-47fe-a319-bc7f78a7a7fa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:46.268348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:46.269190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:46.269612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:46.270053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:46.283423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:46.285212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246b35-be01-4208-a88e-91570a30645b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:46.294730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Mon Aug 1 22:28:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:28:46.302883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:46.316059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6a4ef-1641-4d11-97a3-4cd32f6fd03b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:46.321887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:46.322511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:46.322789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:46.323056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:46.324553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:46.326424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dde37-86ac-47f8-91bb-20a1cf43a0e2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:46.327951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Mon Aug 1 22:28:46 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:48.362008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7145c8-7de4-47dc-a846-9a6b74c2e4d3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.362534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7145c8-7de4-47dc-a846-9a6b74c2e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.362534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7145c8-7de4-47dc-a846-9a6b74c2e4d3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.363828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:48.369035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.369478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.369593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.389338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:48.390548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:48.397751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad467b-3b91-4a65-98e5-6e7d2a36b4e3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.398051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad467b-3b91-4a65-98e5-6e7d2a36b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.398321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad467b-3b91-4a65-98e5-6e7d2a36b4e3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.399633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:48.404950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.405243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.405524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.451618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80398) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 22:28:48.452125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80398) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 22:28:48.472653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:48.494817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:48.497710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:48.499319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:48.519436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:48.598919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:28:48.599568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf05b5c-6738-4b83-8b01-4a1778e0664e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:48.621244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:48.635879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:48.636544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.636828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.637106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.648564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:28:48.657614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:28:48.658553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:28:48.659621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:28:48.664660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:48.721347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61285ed-29bc-4777-8ebb-da5700845c56 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:48.723777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:48.744511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:48.758871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:48.759785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.760064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.760335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.763445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16557dda-0719-459b-83c6-97d7e013313a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:48.796649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:48.797275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:48.797574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:48.797860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:48.798631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:48.798933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:48.800339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:48.813618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:28:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:48.819229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:48.826080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:48.827577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:48.930736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:28:48.936632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:48.940125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:49.032368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83001139-1747-4367-82e1-7496c17f8c78 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:49.044772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54dfe-720b-4799-afea-c64515680c83 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:49.065954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:49.066610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:49.067172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:49.067461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:49.068650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:49.070199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:49.071825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:28:48 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:49.077264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:49.078881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:49.079818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:49.080093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:49.080361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:49.091900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:49.099256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:49.100735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:28:49.119211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:28:49.120068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:28:49.120880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:28:49.120880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:28:49.120880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:28:49.120880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:28:49.120880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 9/18] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:28:49.126426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:49.177121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502c719-27fc-47d0-a226-cb3db05411c1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:49.182281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:49.213353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:49.214042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:49.214372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:49.214610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:49.215841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:49.217330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:49.219347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 10/19] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:49.225209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:49.227116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733a6f5-e936-48f3-8bd4-549f60d78f0f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:49.261967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:49.262671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:49.262959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:49.263234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:49.263935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:49.264239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:49.265665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fff6c1-32fb-4890-ad4b-8d5e203c4ad2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:49.283821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 10/20] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:49.327016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:49.349721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f500e-fc55-4d69-b0da-9257ffcfec0e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:49.388158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:49.388892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:49.389035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:49.389353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:49.390264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.domain.id=default, user.id=47de1e2ad35749d7a331fc838348d562) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:49.390555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:49.392080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad969f85-4e56-4b05-8e07-91f84206f343 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:49.399701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 11/21] 10.222.0.47 () {56 vars in 1322 bytes} [Mon Aug 1 22:28:49 2022] GET /identity/v3/role_assignments?user.id=47de1e2ad35749d7a331fc838348d562&scope.domain.id=default&role.id=c512669c168542d587f922137072be5f => generated 233 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:51.491734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7b5-2058-446f-8a49-79dbf820c252 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:51.492257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7b5-2058-446f-8a49-79dbf820c252 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:51.492257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7b5-2058-446f-8a49-79dbf820c252 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:51.493659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 11/22] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:51.499319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:51.500164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:51.500641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:51.529852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:51.531385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:51.618724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 12/23] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:51.654475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f7b5-2058-446f-8a49-79dbf820c252 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:51.778876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f7b5-2058-446f-8a49-79dbf820c252 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:51.800136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:51.821029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:51.822090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:51.822560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:51.823031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:51.838985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e4635702-977a-4cf1-8af9-588f342bebe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:28:51.846337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:28:51.847188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:51.847959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:28:51.854350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 12/24] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:28:51.860482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:51.945437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec242c-897b-42e6-be6c-36d3008d807d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:51.945803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec242c-897b-42e6-be6c-36d3008d807d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:51.946035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec242c-897b-42e6-be6c-36d3008d807d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:51.947410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 13/25] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:51.953382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a4fc9-6a6c-45d8-9755-29c328aac778 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:51.953700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a4fc9-6a6c-45d8-9755-29c328aac778 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:51.953985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a4fc9-6a6c-45d8-9755-29c328aac778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:51.982124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:51.990633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-af1a4fc9-6a6c-45d8-9755-29c328aac778 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:51.991841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-af1a4fc9-6a6c-45d8-9755-29c328aac778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:52.065824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60706c-ccc7-4c99-8b0b-8778af85c50f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.105086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 14/26] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:28:52.106489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a8eff-6edd-47d3-a308-e99aaca4b325 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.106780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a8eff-6edd-47d3-a308-e99aaca4b325 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.107058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a8eff-6edd-47d3-a308-e99aaca4b325 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.108358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 15/27] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:52.115249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.115543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.115826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.133540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.134217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.134500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.134776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.135517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:52.135818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.137539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.142428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:52.143640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:52.158140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 13/28] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:52.167834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.294696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726fb5e-625d-447c-9fa5-ccbef4ba3792 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.336443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.337425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.337767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.338299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.339503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.340960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a688c-bac0-4e96-80ee-2b7ec84ca7b5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.342100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 16/29] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:52.342596 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 14/30] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:28:52 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:52.344284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d008b-0e5e-4b3d-aa45-459b1c4beb7c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.344571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d008b-0e5e-4b3d-aa45-459b1c4beb7c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.344872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d008b-0e5e-4b3d-aa45-459b1c4beb7c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.346429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 15/31] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:52.347944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.351492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d008b-0e5e-4b3d-aa45-459b1c4beb7c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.356138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:52.389217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:52.459475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e2ce-64a2-4b6a-920f-425710d14ac1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.486786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d008b-0e5e-4b3d-aa45-459b1c4beb7c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.500907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.502218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.502873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.503746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.504884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.507072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.526243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.526897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.527281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.527681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.528954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.530496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef85f0-60fe-4c3e-9982-687753882da5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.532258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 16/32] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:52.533829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.534125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.534388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.553901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:52.555208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:52.632002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 17/33] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:52.633065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.635504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:52.652263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 17/34] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Aug 1 22:28:52 2022] POST /identity/v3/roles => generated 219 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:28:52.653364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.750014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b17e6-8ebf-4209-a15b-f6db6dcb3c5d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.788307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.789326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.789837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.790306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.791369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.793200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.847375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:52.855177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e58a6-13b8-493a-8d7b-6faa2028c395 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:52.901569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:52.902422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:52.902957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:52.903244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] PATH_INFO: `/v3/domains/default/users/47de1e2ad35749d7a331fc838348d562/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:52.909273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=47de1e2ad35749d7a331fc838348d562, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:52.911237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b5400-c0ee-40f3-9d64-75e981ecd302 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:52.930652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 18/35] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Aug 1 22:28:52 2022] POST /identity/v3/roles => generated 225 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:28:52.932081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:52.939690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 18/36] 10.222.0.47 () {58 vars in 1361 bytes} [Mon Aug 1 22:28:52 2022] PUT /identity/v3/domains/default/users/47de1e2ad35749d7a331fc838348d562/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:28:53.020156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:53.062135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32877ce2-9024-4b91-97f3-a81b523eabc1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.102058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.102781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.103070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.103346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.116233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.118058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.135379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1ded-37ed-4159-bf45-79d7122fccc0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.135702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1ded-37ed-4159-bf45-79d7122fccc0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.135933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1ded-37ed-4159-bf45-79d7122fccc0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:28:53.136269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:28:53.137222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:28:53.138056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:28:53.138056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:28:53.138056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:28:53.138056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:28:53.138253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 19/37] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:53.138316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 19/38] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:28:53.143352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.143642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.143919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.147472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:53.178821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:53.180345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:53.257963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5faf-8a73-413c-a606-f9fc7ce15039 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.263165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 20/39] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:53.276426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:53.289321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:53.310044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.310692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.310978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.311254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.311972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:53.312281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.313801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.341329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 20/40] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:53.348222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:53.400014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:53.417682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eacb21-73d4-421f-9e9d-b7b684989185 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.452429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61a8af-0a47-4180-967a-f25f87c387ce None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.454752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.456432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.456864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.457287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.458646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.460683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59f5f4-a560-4470-a712-bc05c774fe7a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.462978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 21/41] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:53.464574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643596-49ce-4995-a1bf-cae9db2ca768 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.465269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643596-49ce-4995-a1bf-cae9db2ca768 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.465703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35643596-49ce-4995-a1bf-cae9db2ca768 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.467235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 22/42] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:53.472933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35643596-49ce-4995-a1bf-cae9db2ca768 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:53.491392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.492250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.492381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.492575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.493269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.494780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaccd4e-0e3f-43a8-95f2-2aa22cf88585 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.537593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 21/43] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:28:53 2022] POST /identity/v3/projects => generated 301 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:28:53.537593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.537593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.537593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.556092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:53.557771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:53.583040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:53.605169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35643596-49ce-4995-a1bf-cae9db2ca768 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.642833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 22/44] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:53.649787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.649787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.649787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.649787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.650128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.650128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45627d-9a98-4303-8944-b30a04b94c3f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.667747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:53.683310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 23/45] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:28:53 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:28:53.798935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db223b5c-330d-4a2a-82cc-7b3f2adeb9dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:53.839198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:53.839852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:53.840138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:53.840414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:53.841151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:53.842642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c746a5-f03c-4f4f-9238-0d5426cc00d9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:53.861640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 23/46] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Aug 1 22:28:53 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:28:55.109034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9704-b25f-4d1a-af98-3d549988f626 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.109626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9704-b25f-4d1a-af98-3d549988f626 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.109626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9704-b25f-4d1a-af98-3d549988f626 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.111038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 24/47] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:55.116490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.116883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.117044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.135900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:55.137104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:55.148105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.207360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 24/48] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:55.230411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb9704-b25f-4d1a-af98-3d549988f626 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.240562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.338959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb9704-b25f-4d1a-af98-3d549988f626 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.374569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.375227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.375511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.375788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.386465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:28:55.392018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:28:55.392902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:28:55.393796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 25/49] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:28:55.399426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.500439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbebed3-cc4e-446b-9a96-b518101abe24 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.535343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.535976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.536258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.536539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.537599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:55.537928 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:55.539340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:55.547661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 25/50] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:55.553711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.620128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.657952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb627c-e437-4074-8ec8-cb5768885d65 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.692347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.693004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.693288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.693579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.694784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:55.696269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0b75f-52de-40a5-9168-f81f582f5269 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:55.697934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 26/51] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:55.703570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.716039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ebf7-ec05-4eab-8a24-18da8abebff3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.716336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ebf7-ec05-4eab-8a24-18da8abebff3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.716655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ebf7-ec05-4eab-8a24-18da8abebff3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.716803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.718087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 27/52] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:28:55.723532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.723807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.724089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.742244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:55.743455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:55.806313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234a60a-5405-4be3-8ad0-eccd799d4769 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.809486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 28/53] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:28:55.831825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.838843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.839566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.839774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.840054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.841204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:55.842718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:55.844695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 26/54] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:55.851837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.899422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.932157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:55.936114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6eb034-3443-405d-be59-2763ad5013a5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.954756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143d04d-7bc3-4cea-b644-a9e315cd5ad4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:55.970308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.970967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.971253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.971533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.972682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:55.974191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:55.975813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 29/55] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:55.981413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:55.987986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:55.988607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:55.988889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:55.989162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:55.989990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.domain.id=default, user.id=47de1e2ad35749d7a331fc838348d562) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:28:55.990292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:55.991741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c853f6-0ea4-4f5c-b537-15cfb04682ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:55.997763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 27/56] 10.222.0.47 () {56 vars in 1322 bytes} [Mon Aug 1 22:28:55 2022] GET /identity/v3/role_assignments?user.id=47de1e2ad35749d7a331fc838348d562&scope.domain.id=default&role.id=c512669c168542d587f922137072be5f => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:28:56.092992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692fd2d-218e-484b-b8e2-a68730918619 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:56.125863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:56.131683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:56.132332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:56.132615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:56.132894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:56.133622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:56.135112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63474f14-ae41-4fe8-b6a7-81e4085002f3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:56.167067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 30/57] 10.222.0.47 () {60 vars in 1140 bytes} [Mon Aug 1 22:28:55 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:28:58.150725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbcd95-39f7-4584-851a-4f3bed10c262 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:58.151533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbcd95-39f7-4584-851a-4f3bed10c262 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:58.151669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbcd95-39f7-4584-851a-4f3bed10c262 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:58.153907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 28/58] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:28:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:28:58.159231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:58.159641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:58.159763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:58.179825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:28:58.181022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:28:58.253061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 31/59] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:28:58.276998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbcd95-39f7-4584-851a-4f3bed10c262 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:58.391618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbcd95-39f7-4584-851a-4f3bed10c262 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:58.426263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:58.426892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:58.427169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:58.427437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:58.428691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:58.430184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da404ee-03c3-4641-95fa-805b3952950d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:58.431720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 29/60] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:28:58.438168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:28:58.542429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061f309-973f-460b-ba81-ffbf27251cc8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:28:58.556312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:28:58.578087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:28:58.578705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:28:58.578988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:28:58.579297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:28:58.580013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:28:58.581491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b599-4f16-49d5-91b8-fc74526a99e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:28:58.611997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 32/61] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:28:58 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:01.595160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9cab-aa72-4847-87a3-1626b10731ff None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:01.596201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9cab-aa72-4847-87a3-1626b10731ff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:01.596684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e9cab-aa72-4847-87a3-1626b10731ff None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:01.598312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 30/62] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:01.603691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:01.604490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:01.604964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:01.634584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:01.636148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:01.703005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979b59d-f071-4731-bf48-2551a2be96ee None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:01.703654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979b59d-f071-4731-bf48-2551a2be96ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:01.704124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979b59d-f071-4731-bf48-2551a2be96ee None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:01.705761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 31/63] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:01.718137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167205-4278-48be-9348-b17433b744f7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:01.719060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167205-4278-48be-9348-b17433b744f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:01.719496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 33/64] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:01.719730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167205-4278-48be-9348-b17433b744f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:01.744333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:01.758729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b3167205-4278-48be-9348-b17433b744f7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:01.760236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b3167205-4278-48be-9348-b17433b744f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:01.766042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:01.894901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 32/65] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:01.908478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:01.919801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564f9f-425e-499d-8fa8-0487a3544553 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:01.925139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3167205-4278-48be-9348-b17433b744f7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:01.976244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:01.976920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:01.977207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:01.977485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.010560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:02.015699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:02.016776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:02.017647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:02.025712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 34/66] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:02.032303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3167205-4278-48be-9348-b17433b744f7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.034573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.068310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.069207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.069653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.070059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.081176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.082352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-23bcc056-e3da-43ae-b4aa-52184c8311be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:02.087416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:02.091033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:02.091811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 33/67] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:02.094616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b3a9f-86ab-4c74-9a93-1d913fa8f2ee None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.095173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b3a9f-86ab-4c74-9a93-1d913fa8f2ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.095666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b3a9f-86ab-4c74-9a93-1d913fa8f2ee None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.097318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 34/68] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:02.098474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b3a9f-86ab-4c74-9a93-1d913fa8f2ee None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.176790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be342c-ab11-499a-8baf-640156c54373 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.215280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.215938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.216217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.216488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.217214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:02.217515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.219455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc08-9c40-41cb-848a-5c6c240d2e7e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.227670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b3a9f-86ab-4c74-9a93-1d913fa8f2ee None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.234303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 35/69] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:02.239871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.239871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.239871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.260026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:02.261209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:02.282124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.282516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.283503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.284296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.284776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.286069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:02.286848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.289046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.301393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 35/70] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:02.302739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.348604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 36/71] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:02.349922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.373394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.415869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ea92-cd9a-4bed-8a76-b67a8248e9cf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.452995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.454000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.454404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.454817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.462214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba14be-9380-47ef-85b8-b592ff24f732 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.469128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.470699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca994-a316-4817-ba31-1bf1e8357aa1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.472389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 36/72] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:02.473580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5bcfc-1403-4a05-994c-72651ccef759 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.473868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5bcfc-1403-4a05-994c-72651ccef759 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.474145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5bcfc-1403-4a05-994c-72651ccef759 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.475515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 37/73] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:02.476047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5bcfc-1403-4a05-994c-72651ccef759 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.500237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.500896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.501238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.501516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.502866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.504355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f833c44-a9dd-4a94-acc0-9b869cd8ec74 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.506346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 37/74] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:02.507584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17003be-249b-4ca9-a5a9-c8299cf56568 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.507871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17003be-249b-4ca9-a5a9-c8299cf56568 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.508150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17003be-249b-4ca9-a5a9-c8299cf56568 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.509505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 38/75] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:02.510152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17003be-249b-4ca9-a5a9-c8299cf56568 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.532902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.580420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5bcfc-1403-4a05-994c-72651ccef759 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.600141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.617416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.618103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.618384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.618474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17003be-249b-4ca9-a5a9-c8299cf56568 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.618873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.628752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2718899c-51a9-42ff-bff5-eceaa9ad2fc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:02.633732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:02.634637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:02.635442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 38/76] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:02.636075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.636291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.636686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.654035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:02.654323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.654958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.655259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:02.655391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.655475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.733755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 39/77] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:02.733755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.734030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.734030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476858-9f2e-46a7-837c-a0cdf3cd01e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.745379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.820919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 39/78] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:02.822206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.822492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.822768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.830601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e7c6-78b6-479a-8e06-0b14dfe6d1cd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:02.839681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:02.840047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:02.841331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:02.865437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:02.866099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:02.866381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:02.866652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:02.907335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 40/79] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:02.908109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:02.934648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:02.936138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:02.955799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.008173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac216a3-fc7a-4ce2-a1a9-661093a595f2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.036801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 40/80] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:03.037717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.044204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.044838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.045114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.045385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.046151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:03.046450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.047913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.048080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.056442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 41/81] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.058377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.138369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a831b2-f3cc-4b42-822d-121768be88e0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.158223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3449-8a54-45ba-87fd-9e52d23e07ba None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.170958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.171133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.171603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.171882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.172516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.182571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:03.187496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:03.188388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 41/82] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:03.194436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.195168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.195168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.195550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.195718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.196945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.198436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.200108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 42/83] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.200864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.262783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.295367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ad170-4454-4aca-a29c-d0f66325942b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.303795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272531d-224e-4f4b-adee-5cc3975da344 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.328459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.329098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.329424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.329730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.330876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.332345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.338459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 42/84] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.338459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.340740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.341404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.341712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.341990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.343251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.345195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.346936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 43/85] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.347692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.399949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.440317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b8a95-ad5c-49d9-b477-d6c36195efd3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.451424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bfc09-8b41-4b2b-be4e-1dc0ec77bd2b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.473964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.474616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.474904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.475188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.476368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.477888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.479560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 43/86] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.480323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.483597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.485106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.485937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.486228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.486491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.487254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:03.487560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.488964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.497275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 44/87] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.498074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.580598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319748d-a968-4d96-b26f-94d80c35bba3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.598131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c014c-1d50-4995-8b2f-7251511bd6f2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.608157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.615465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.616101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.616385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.616658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.617463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:03.617808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.619284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.625264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 44/88] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.626203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.637159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.637911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.638132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.638408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.639150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.640653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.666087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 45/89] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:03 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:03.666974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.721625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:03.764195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3cf6-43e5-4dd6-b13f-0937cb6feaa7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.803306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.805137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.805137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.805318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.806115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=47de1e2ad35749d7a331fc838348d562) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:03.806426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:03.807932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:03.815061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 45/90] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/role_assignments?user.id=47de1e2ad35749d7a331fc838348d562&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:03.815934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:03.865464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d785f10-b404-4aaa-a2c6-63d06a015a59 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.919646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:03.920306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:03.920590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:03.920965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:03.975351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54b4f-11b3-4dfc-b2ac-b32d1cd5e778 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:03.976058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.022588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.023059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.025398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.029712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.030481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.030837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.031233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.033788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.035362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.037088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 46/91] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:04.043379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:04.182584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5aa5f-9832-4bee-afee-6c92679a7ca7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:04.186754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 46/92] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:04.192440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:04.225038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.225863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.226438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.226735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.277712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.320638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.323368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.325411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.328256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcdeac-6fea-47a4-8c12-84c1bb6ca5db None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:04.373240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.373937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.374221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.374957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.376595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.378401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.381281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 47/93] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:04 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:04.387407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:04.443921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 47/94] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:04.449533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:04.507661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6dc25-a1c6-4416-834f-7d6c8c924f27 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:04.544985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.545654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.546041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.546380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.547190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:04.547545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.549279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.551633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.564635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 48/95] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:04 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=0faca02de5824306b70c798c90d5f033 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:04.575145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:04.578578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dcf5e-049b-4388-9a3d-7ef6f2f6c03c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:04.630348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.631143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.631296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.631556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] PATH_INFO: `/v3/projects/72e92d38e374440e893b711ce60d9320` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.632769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] RBAC: Authorizing `identity:get_project(project_id=72e92d38e374440e893b711ce60d9320)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.634294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef995a-c83f-44c6-b57a-ae3111935adc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.636032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 48/96] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:04 2022] GET /identity/v3/projects/72e92d38e374440e893b711ce60d9320 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:04.642502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:04.763033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a5a84-6ecc-4803-a436-4a93d028886d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:04.806026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:04.806708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:04.806989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:04.807267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:04.808085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=72e92d38e374440e893b711ce60d9320, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:04.808394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:04.809930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e504b-e747-489e-b8ef-a59c99ff7687 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:04.822576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 49/97] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:04 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=72e92d38e374440e893b711ce60d9320&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:06.466114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57961ba1-4c16-4e09-8f03-7c0cfd284bed None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:06.466660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57961ba1-4c16-4e09-8f03-7c0cfd284bed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:06.466660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57961ba1-4c16-4e09-8f03-7c0cfd284bed None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:06.468057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 49/98] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:06.473874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b8fe-4a76-4d7b-9fce-20b099e9e84e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:06.474273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b8fe-4a76-4d7b-9fce-20b099e9e84e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:06.474449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b8fe-4a76-4d7b-9fce-20b099e9e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:06.498956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6385b8fe-4a76-4d7b-9fce-20b099e9e84e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:06.500223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6385b8fe-4a76-4d7b-9fce-20b099e9e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:06.575981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:06.587095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 50/99] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:06.618502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57961ba1-4c16-4e09-8f03-7c0cfd284bed None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:06.714069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38659c-decd-4691-aa1f-cc380dccb227 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:06.714404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38659c-decd-4691-aa1f-cc380dccb227 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:06.714636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38659c-decd-4691-aa1f-cc380dccb227 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:06.716020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 51/100] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:06.722815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:06.723071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:06.723341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:06.754446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:06.755664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:06.796458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57961ba1-4c16-4e09-8f03-7c0cfd284bed None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:06.851364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:06.852497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:06.852966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:06.853426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:06.858312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:06.859973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:06.861958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 50/101] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:06.865946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 52/102] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:06.867430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:06.906977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:06.909042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.051942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36790589-0f9b-4eb5-86b0-bf044e01c6ff None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.077965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df79a-d1d4-4eec-80c3-6b955961d531 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.102963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.120525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.121306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.121806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.122196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.123155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:07.125040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813233fb-cddb-4156-b8c0-35a0ed9a73f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:07.125764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.126680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.127136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.127579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.159006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2f6507e2-999e-4ce0-8536-eace4e565598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:07.164958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:07.165829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:07.166592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:07.172473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 53/103] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:07.176407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04c9c5-d535-4da2-be94-cb89d4d663be None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.176407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04c9c5-d535-4da2-be94-cb89d4d663be None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.176407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04c9c5-d535-4da2-be94-cb89d4d663be None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.176407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 54/104] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:07.177216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc532fd-bf6c-4918-afb2-f128ba97b6e5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.177500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc532fd-bf6c-4918-afb2-f128ba97b6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.177807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc532fd-bf6c-4918-afb2-f128ba97b6e5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.179231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 55/105] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:07.181775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 51/106] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:29:06 2022] POST /identity/v3/projects => generated 302 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:29:07.185575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.185575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.185575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.191971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.192485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.192941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.228564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:07.230107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:07.234552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:07.236109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:07.364539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 56/107] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:07.365343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.380852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 52/108] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:07.396097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.432318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.522950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630179f9-160d-42b5-ad1e-b4a2e43161c6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.531116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d97b6f-f959-4fd2-a58e-5d2b5e2bada6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.537210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.560331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.561341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.561841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.562326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.563245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:07.563881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:07.565742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:07.567517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.568250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.568626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.569047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.576253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 57/109] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:07.577307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.581678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:07.586205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:07.587058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:07.587814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 53/110] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:07.593430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.682023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.705332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012d4f3-e48d-467d-b1e8-5150d5ba47df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.716119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d5268-2955-4c0d-8642-fe7e4ea8e21e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.742989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.743951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.744405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.744850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.745795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:07.746273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:07.747870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:07.757318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 54/111] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:07.758402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.763635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.764460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.764819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.765156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.777217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6bdc527d-8ad1-4d16-917f-7154fe047dbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:07.783050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:07.783909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:07.784655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 58/112] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:07.786052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b455c-02c2-4a2a-a96a-6588a43fa9c0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.786151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b455c-02c2-4a2a-a96a-6588a43fa9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.786739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b455c-02c2-4a2a-a96a-6588a43fa9c0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.787690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 59/113] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:07.788486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b455c-02c2-4a2a-a96a-6588a43fa9c0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:07.826656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.875467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf59180-42b2-480f-bfe3-251d94dd8bb8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.898830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b455c-02c2-4a2a-a96a-6588a43fa9c0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:07.911159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.911840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.912125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.912399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.913646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:07.915151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fc6ed-eb99-4a4f-b51a-47e539f87815 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:07.916778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 55/114] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:07.917982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64845658-1350-459d-be88-9c1d49ba6bab None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.918247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64845658-1350-459d-be88-9c1d49ba6bab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.918529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64845658-1350-459d-be88-9c1d49ba6bab None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.919728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 56/115] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:07.922791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.923144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.923425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.939690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:07.940767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:07.941262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:07.941369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:07.943187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:07.943310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:07.945002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:07.945232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:07.946755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:07.948501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 60/116] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:07.949297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.021148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 57/117] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:08.021888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.058193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e16b-248d-4498-b592-b677b7f0c2e9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.059834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.097131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.097942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.098226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.098499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.099334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:08.099642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.101020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2756ece-66dc-4243-bb7c-dd2ba5e05a98 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.110528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 61/118] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:08.111749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.112005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.112311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.130677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:08.132012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:08.135820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7241c26-bd9d-421d-b579-794d9645ae25 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.171637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.172408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.172698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.173055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.174296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.175771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.177585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 58/119] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:08.178265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.187516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.206894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 62/120] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:08.207592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.297092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c52b-b51f-4275-ba9e-36339a30f756 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.313108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.320710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78cec-bd7f-4637-98b5-7a4863604c50 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.332340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.332998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.333291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.333631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.334755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.336231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.343042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 59/121] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:08.344230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.357895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.358535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.358814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.359080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.370025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:08.374995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:08.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:08.376668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 63/122] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:08.377310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.430321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.455585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7163d-b90d-43ee-8d0e-ed398b45da4e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.487293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb44eb-9e25-423e-9f95-f5b5a2766314 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.491607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.492268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.492550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.492829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] PATH_INFO: `/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.507927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] RBAC: Authorizing `identity:get_user(user_id=13f4e3bea1e041c9bdd3ac55d9589c5b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.509529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.511282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 60/123] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:08.512194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.529732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.531146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.531651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.532207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/47de1e2ad35749d7a331fc838348d562/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.534909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, user_id=47de1e2ad35749d7a331fc838348d562, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.536808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.553813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 64/124] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:08 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/47de1e2ad35749d7a331fc838348d562/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:08.555145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.583454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.633592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faad39-ad15-4aeb-9687-39217b16761b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.667239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.672903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.673761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.674214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.674625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.687041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:08.693020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:08.696338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:08.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 61/125] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:08.700713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.706119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf40f3d-bb51-4a5d-9ac9-f124a2901758 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.760513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.761651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.762056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.762433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/e7877688e78b41359eb7ae4a89ec65df/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.765284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, user_id=e7877688e78b41359eb7ae4a89ec65df, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.766918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.788805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 65/126] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:08 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/e7877688e78b41359eb7ae4a89ec65df/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:08.789825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.848294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3da60-5549-40ae-ab6c-a1122662a8f0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:08.930878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:08.931520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:08.931797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:08.932072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:08.932808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:08.933112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:08.934790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:08.937851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:08.951439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 62/127] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:08.952337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:08.994995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ff2dd-adf4-453c-96e3-f37f6f743901 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.031849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.043011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.043918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.044396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.044835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.091455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1c18d-d5ea-472c-a683-20684dcccfad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.131094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.132099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.132600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.133061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.134040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:09.134586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.136159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.137425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.139611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.146666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 63/128] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.147890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.233985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.265476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 66/129] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:09.270653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.347321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3355d-3b7e-44d6-bb8c-f26a52a27d35 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.367357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.389208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.389966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.390253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.390582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.391904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.393372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.395625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 64/130] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.396477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.452186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac4bb-2302-4a03-9436-f245a25627a1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.512582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.513500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.513844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.514125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.515615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.517783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.521194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 67/131] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.522119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.555997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc168684-0feb-46a3-934e-78fcc6fa068a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.579325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.592671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.593438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.593772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.594412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.595634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.597366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3423cea-6eda-4486-ac5f-37708de89c68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.599190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 65/132] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.600541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb46b-bb90-4a85-a3ad-a8ddece52a3f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.600832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb46b-bb90-4a85-a3ad-a8ddece52a3f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.601109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb46b-bb90-4a85-a3ad-a8ddece52a3f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.602623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 66/133] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:09.603184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb46b-bb90-4a85-a3ad-a8ddece52a3f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.650037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa4dd-c52d-475f-87fa-ce084a186df9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.675450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.691926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.692615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.692917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.693209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.694578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.696009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.697750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 68/134] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.698519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.748100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb46b-bb90-4a85-a3ad-a8ddece52a3f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.796680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.798510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.799292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.799812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] PATH_INFO: `/v3/projects/72e92d38e374440e893b711ce60d9320` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.802221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] RBAC: Authorizing `identity:get_project(project_id=72e92d38e374440e893b711ce60d9320)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.803961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3a791-a113-485a-a1f7-164dcafdb373 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.804088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5272b4-dbc5-41da-9a28-b2fef4394908 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:09.807255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 67/135] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/projects/72e92d38e374440e893b711ce60d9320 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.808806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.809307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.809806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.820853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:09.833797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:09.835007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:09.856232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:09.857490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:09.858047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:09.858761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:09.860359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=13f4e3bea1e041c9bdd3ac55d9589c5b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:09.860972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:09.863363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:09.870994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 69/136] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:09 2022] GET /identity/v3/role_assignments?user.id=13f4e3bea1e041c9bdd3ac55d9589c5b&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=3d10e61b117d4ae5986d910045db8bea => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:09.872933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.907856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 68/137] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:09.908947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:09.977463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:10.040890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae5036-7c72-4486-bdae-d0a35a8fd98d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:10.081613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:10.082378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:10.082676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:10.082950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] PATH_INFO: `/v3/projects/72e92d38e374440e893b711ce60d9320/users/e7877688e78b41359eb7ae4a89ec65df/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:10.084976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] RBAC: Authorizing `identity:create_grant(project_id=72e92d38e374440e893b711ce60d9320, user_id=e7877688e78b41359eb7ae4a89ec65df, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:10.086704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:10.110246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 69/138] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:09 2022] PUT /identity/v3/projects/72e92d38e374440e893b711ce60d9320/users/e7877688e78b41359eb7ae4a89ec65df/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:10.111157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:10.114882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e819720-9b72-4af1-b550-1c0ba39d24be None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:10.194571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:10.195290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:10.195579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:10.195862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/e7877688e78b41359eb7ae4a89ec65df/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:10.203406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, user_id=e7877688e78b41359eb7ae4a89ec65df, role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:10.204994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:10.242413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 70/139] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:09 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/e7877688e78b41359eb7ae4a89ec65df/roles/0faca02de5824306b70c798c90d5f033 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:10.273454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc14a-71f9-4c82-b595-1dade7827b85 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:10.293695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:10.317109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:10.317898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:10.318185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:10.318511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:10.319737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:10.321288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526867e-6d70-4a28-bf73-8081d0f20203 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:10.323341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 70/140] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:10.329486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:10.385654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:10.442729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c50005-9e35-41cf-8183-943c7fc661c9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:10.495652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:10.496689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:10.497184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:10.497685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:10.498591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:10.500330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc9fba-4f8d-4d28-a4ac-2bb676106247 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:10.526937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 71/141] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:10 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:29:11.268088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9de-b8ef-4b9c-a6ed-8f13777c0703 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.268626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9de-b8ef-4b9c-a6ed-8f13777c0703 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.268626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9de-b8ef-4b9c-a6ed-8f13777c0703 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.270098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 71/142] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:11.276522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399917-1818-4940-a9d4-39c2bfee5f8a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.277405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399917-1818-4940-a9d4-39c2bfee5f8a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.277914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399917-1818-4940-a9d4-39c2bfee5f8a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.279748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 72/143] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:11.279867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.280060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.280330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.285479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.285801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.286084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.305760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:11.306980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:11.310589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:11.311753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:11.411796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 73/144] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:11.427009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 72/145] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:11.472581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:11.487227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:11.498360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:11.637402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e823-6c1c-421c-98bf-a518edce812d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:11.659413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91673933-25f9-41ee-9fc4-8a0b46e5147e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:11.671852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:11.702713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:11.703389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.703700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.703966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.721407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:11.724379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:11.725056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.725338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.725649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:11.729948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:11.730833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:11.732349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 74/146] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:11.738302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:11.758549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:11.764639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:11.765513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:11.766324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:11.770277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 73/147] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:11.776018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:11.861932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:11.879849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ad8d1-43ff-44ee-b084-ced42a9b6281 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:11.921670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:11.922239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:11.922522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:11.922799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:11.923686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:11.923997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:11.925408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:11.935569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 75/148] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:11.940802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.058371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dcdc-5e31-459d-a5ba-81cc1984347e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.066773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8759-0d3b-4982-8f98-1a62436cfa93 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.099769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.100435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.100718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.100991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.102267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.103781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.105841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 76/149] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:11 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:12.110507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.111154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.111435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.111708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.112484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:12.112752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.114977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.119942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.136316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.139047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 74/150] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:12.144681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.210025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.271876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec37f28-e70a-46c8-b3b8-617481bc5cd9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.318738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.319395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991e8b-dd92-4ab8-b3e5-8ed39cd8041b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.319733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.319733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.320051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.390307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.390987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.391273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.391542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] PATH_INFO: `/v3/users/47de1e2ad35749d7a331fc838348d562` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.396741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] RBAC: Authorizing `identity:get_user(user_id=47de1e2ad35749d7a331fc838348d562)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.398275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.400234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 75/151] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/users/47de1e2ad35749d7a331fc838348d562 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:12.405330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.415918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.417678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a5a74-8ec9-4e01-be8d-bd7f2ac0c104 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.437947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.550080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.555324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92e9-820d-4906-8046-a8db49baa180 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.563825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 77/152] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:12.565247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24631c-a618-4484-b78b-5ce650250262 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.565537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24631c-a618-4484-b78b-5ce650250262 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.570171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24631c-a618-4484-b78b-5ce650250262 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.571699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 78/153] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:12.576471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24631c-a618-4484-b78b-5ce650250262 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.631003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.631642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.632017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.632287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.734287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24631c-a618-4484-b78b-5ce650250262 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:12.745916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.746812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.748319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.788929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:12.789627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.789909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.790185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.794357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:12.795849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049d043-4568-4ddd-9b5e-ab0016a9b84d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:12.798143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 79/154] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:12.799322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:12.799605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:12.799877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:12.834747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:12.835885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:12.893973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:12.900840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 76/155] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:12.901843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:12.940165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 80/156] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:12.941095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.062169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7107-4f42-4742-b9fc-d8d074ebc541 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.092538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5cb3-31e4-4b1e-ad7a-5436772cabf4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.126448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.127094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.127369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.127636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.128440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:13.128743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:13.129908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:13.130233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:13.140812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 77/157] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:13.141516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.151380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.152076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.152358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.152637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.158059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:13.159572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:13.161839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 81/158] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:12 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:13.167722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.246025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:13.309319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae9a34-6588-4db9-a7b3-c1e21e16f809 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.327234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1fb9d-9684-4b55-8440-9b5f40131522 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.346915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.348148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.348568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.348993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.349997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=47de1e2ad35749d7a331fc838348d562) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:13.350461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:13.352136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:13.359129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 82/159] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/role_assignments?user.id=47de1e2ad35749d7a331fc838348d562&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:13.424403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.425080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.425359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.425674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.450609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:13.456187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:13.457062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:13.457903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 78/160] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:13.463740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.534039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:13.590009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b3811-052b-4e9c-be25-afe3ed8d19a8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.628488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.629462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.630095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.630577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.631522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:13.632046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:13.633827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698325-e5da-4d1a-b021-a4bf7006ca7b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:13.643190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 83/161] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:13.649032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.799725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032c7ad-0947-4968-9a1e-16ec195260e2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.825411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ce09-a277-4ff2-b14a-6c5acc279a48 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.826117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ce09-a277-4ff2-b14a-6c5acc279a48 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.826605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ce09-a277-4ff2-b14a-6c5acc279a48 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.828218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 84/162] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:13.830191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:13.833921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.834434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.835092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.850937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:13.851602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:13.851879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:13.852150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] PATH_INFO: `/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:13.854258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] RBAC: Authorizing `identity:get_user(user_id=13f4e3bea1e041c9bdd3ac55d9589c5b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:13.855744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:13.857404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 79/163] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:13.865480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:13.865676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:13.867262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:13.961704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 85/164] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:13.989090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfa444-70ae-48ca-85ae-ebdcb1a42505 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:13.993803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.053970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.066309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.066994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.067273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.067546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.074397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:14.075889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:14.077665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 80/165] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:14.083543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.191753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f29d0-e0e3-40fe-ab25-0bb72ffd6838 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.209941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.267043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f8b5-d5b0-4951-98e9-3370e6fb132b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.294859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.295530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.295806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.296073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.319286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.319992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.320272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.320546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/13f4e3bea1e041c9bdd3ac55d9589c5b/roles/3d10e61b117d4ae5986d910045db8bea` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.324130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, user_id=13f4e3bea1e041c9bdd3ac55d9589c5b, role_id=3d10e61b117d4ae5986d910045db8bea)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:14.325639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:14.332921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c44d57ea-32ca-416c-926c-0ff745889829 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:14.338238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:14.339361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 86/166] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:14.340834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fab89-a3dc-463f-b734-4946689a7d16 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.341773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fab89-a3dc-463f-b734-4946689a7d16 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.341773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fab89-a3dc-463f-b734-4946689a7d16 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.349238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 81/167] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:14 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/users/13f4e3bea1e041c9bdd3ac55d9589c5b/roles/3d10e61b117d4ae5986d910045db8bea => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:14.350964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.355126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 87/168] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:14.363097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.363374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.364210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.418909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:14.420144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:14.464503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6775-c323-45dc-ad23-fcb7da379340 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.493995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.512083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.513142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.513699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.514154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.515108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:14.515648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:14.517430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:14.527760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 82/169] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:14.536748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.615333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 88/170] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:14.646669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.650444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081db23-8f31-4a67-9079-8b0928f9dbea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.687893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.689221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.690619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.691109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.691665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.693448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:14.695264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:14.697490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 83/171] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:14 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:14.702571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.774023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.778194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c787d-0197-4de9-b1c7-fcde24c34ef1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.809833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc3e3d-f93f-4454-9526-8a74807cedfd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:14.811719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.812373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.812645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.812933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.823292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-100b9a64-6ccf-42b2-8536-fbaefbc2af2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:14.828760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:14.829716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:14.830495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 89/172] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:14.831683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655419b4-6583-4bce-8ca0-916e419c6087 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.831777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655419b4-6583-4bce-8ca0-916e419c6087 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.832019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655419b4-6583-4bce-8ca0-916e419c6087 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.833461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 90/173] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:14.839008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.839386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.839990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.845652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:14.846298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:14.846626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:14.846910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:14.861521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:14.863330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:14.918151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:14.919653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:14.934244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 91/174] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:14.935221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:14.949425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:14.978962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.039459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 84/175] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:15.040110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417b774-cf8c-4305-ba71-860aade1a725 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.040765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.077583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.077583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.077583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.077583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.078038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:15.078038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.079943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.088947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 92/176] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.090179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.181902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.209866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac9db7-60a0-4da0-ab91-1388bf06b17b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.247368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.248183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.248553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.248918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.250316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.251979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.253848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 93/177] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.254698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.271542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb559563-cb94-4c31-9ce6-f4614a565d7d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.360930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc240c6-090e-41cf-870f-9e10fb671fc2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.398364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.399014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.399285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.399553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] PATH_INFO: `/v3/users/e7877688e78b41359eb7ae4a89ec65df` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.400955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] RBAC: Authorizing `identity:get_user(user_id=e7877688e78b41359eb7ae4a89ec65df)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.402937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.404725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.405384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.405683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.405953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.406944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 94/178] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/users/e7877688e78b41359eb7ae4a89ec65df => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.407820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.425098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:15.433016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:15.433961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:15.434765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:15.440980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 85/179] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:15.441570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.441868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.454921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.555052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2af84c-86f9-4779-befa-0c14e71a6004 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.591632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.592409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.592781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.593162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.671604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.673427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406db277-4f03-4ce7-9f76-6c01331285e1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.673544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d5a86-9f4e-4091-b394-eba87befd17c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.685836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.718874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.719911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.720185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.720449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.721240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:15.721539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.723594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.732215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 95/180] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=0faca02de5824306b70c798c90d5f033 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.733661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.789055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 86/181] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:15.790653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d2924-78f4-498d-8a26-94e707dac22c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.790794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d2924-78f4-498d-8a26-94e707dac22c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.790987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d2924-78f4-498d-8a26-94e707dac22c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.796520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 87/182] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:15.798274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0249-3d9d-4fb1-a884-4304d67c9d25 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.798458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0249-3d9d-4fb1-a884-4304d67c9d25 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.798738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0249-3d9d-4fb1-a884-4304d67c9d25 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.800408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 88/183] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:15.801214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee0249-3d9d-4fb1-a884-4304d67c9d25 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:15.838796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.867703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59115227-91c4-46f4-9fae-7b32fa4eefa7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.905126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.906261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.906261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.906423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.907048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:15.907402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.908790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fe017-680d-4308-8e99-a93406c1f013 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.920714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 96/184] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.922185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.922519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.925446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.937633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee0249-3d9d-4fb1-a884-4304d67c9d25 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:15.945059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:15.946282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:15.949677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:15.983254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:15.984006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.984359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.984699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] PATH_INFO: `/v3/projects/72e92d38e374440e893b711ce60d9320` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:15.986022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] RBAC: Authorizing `identity:get_project(project_id=72e92d38e374440e893b711ce60d9320)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:15.988349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebda6-2b90-4ba6-97e3-ea0e47a815fc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:15.990283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 89/185] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:15 2022] GET /identity/v3/projects/72e92d38e374440e893b711ce60d9320 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:15.991257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:15.991540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:15.991809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.024457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:16.025692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:16.033641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 97/186] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:16.033641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.098832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.103525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 90/187] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:16.117583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.144040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd110-a12f-4eb3-8fe9-e5fc225309d9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.179370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.180255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.180681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.180954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.196122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.197727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.199494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 98/188] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:16.200327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.223623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf80a87-77df-4be4-b912-25ce98558ef8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.231956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.259711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.260342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.260625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.260900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.261731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=72e92d38e374440e893b711ce60d9320, user.id=e7877688e78b41359eb7ae4a89ec65df) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:16.262048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.263591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.269852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 91/189] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/role_assignments?user.id=e7877688e78b41359eb7ae4a89ec65df&scope.project.id=72e92d38e374440e893b711ce60d9320&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:16.270495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.327888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3130-a6c2-4cf4-95d3-f03743b2992f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.364491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.381467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.382698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.383527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.383951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.385627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.386672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24173b-303b-4ec7-ad03-f0b62c8f3afa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.387497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.391088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 99/190] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:16.392334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.433112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.433896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.434351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.434685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.447950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:16.454652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:16.455764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:16.456737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:16.458491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 92/191] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:16.459428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.494076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.532476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4b271-ddb3-40eb-a9df-a310c5440e2c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.570055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.570679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.570953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.571219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.584581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e118af-3758-4aa0-8ef4-b6bcd29c2642 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.618411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.619072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.619356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.619632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.620321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.621820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.635649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.646808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.648339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.652273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 93/192] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:16 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:16.654443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.777380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 100/193] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:16.784801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.786762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.806820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5968ee-54dd-46a8-b10a-548153e4ef42 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.855100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.855874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.856151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.856419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.857153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:16.857451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.858878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.869297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 94/194] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:16.875589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:16.908177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:16.921595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f8a68-c0a4-4b53-86c6-758f398206f4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:16.964163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:16.964888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.965161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.965427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.967732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:16.969546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94a1a-4da8-47f9-bba0-92c862f8ea8e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:16.971392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 101/195] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:16.972745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615bebf-9d34-4271-a8fd-9ca878284604 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:16.972927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615bebf-9d34-4271-a8fd-9ca878284604 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:16.973165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615bebf-9d34-4271-a8fd-9ca878284604 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:16.974809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 102/196] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:16.978287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615bebf-9d34-4271-a8fd-9ca878284604 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.004331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a0c6c-59bf-436b-b0c2-7a4b0378bcbe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.043726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.044369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.045027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.045323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.048699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:17.050943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9c6da-4071-4919-a368-0a2fd36282bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:17.052584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 95/197] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:17.053899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.054347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.054632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.076957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:17.078181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:17.094371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:17.105460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615bebf-9d34-4271-a8fd-9ca878284604 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.147798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.148613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.148959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.149340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.150279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=7bcc153e9e8747149d6e87313c554dca) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:17.150647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:17.152264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:17.159713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 103/198] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:16 2022] GET /identity/v3/role_assignments?user.id=7bcc153e9e8747149d6e87313c554dca&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:17.165686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.197062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 96/199] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:17.213122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:17.239249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.364579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb65ae-8fcd-4fe8-94f7-f7ad0af750e9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.388127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d383e7-42e7-41b4-ab86-c58cb7be555d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.414292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.414930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.415202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.415466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.444394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.445709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.446133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.446545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.470154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:17.478354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:17.479237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:17.480030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 97/200] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:17.487270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.503659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:17.529011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:17.530523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:17.572816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:17.676680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffd841-b85a-4e9f-9e08-d4c8fa154f0f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.715629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 104/201] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:17.733980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.737829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.738474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.738753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.739029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.739760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:17.740062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:17.741480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:17.750417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 98/202] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:17.756164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.830651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:17.874051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ddff-8449-4a5c-9969-ca9043fb6f06 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.928099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11e960-8b67-4fec-b0b4-a83086a35fcb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:17.947746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:17.948417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:17.948697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:17.948971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] PATH_INFO: `/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:17.950215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] RBAC: Authorizing `identity:get_user(user_id=13f4e3bea1e041c9bdd3ac55d9589c5b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:17.951684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:17.953495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 99/203] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/users/13f4e3bea1e041c9bdd3ac55d9589c5b => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:17.972499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:17.986917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:18.041029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:18.041736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:18.042023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:18.042299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:18.046295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:18.047778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:18.049866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 105/204] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:18.062576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:18.095947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56ca3-7960-48de-850c-c2e7da50acb2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:18.137787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:18.138421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:18.140611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:18.140893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:18.150158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:18.266962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812a485-23cf-43a0-9fb4-de00a95ff5c5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:18.308225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:18.308857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:18.309133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:18.309403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:18.310232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=7bcc153e9e8747149d6e87313c554dca) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:18.311082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:18.312601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:18.314961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:18.316411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:18.326705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 106/205] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:18 2022] GET /identity/v3/role_assignments?user.id=7bcc153e9e8747149d6e87313c554dca&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=0faca02de5824306b70c798c90d5f033 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:18.461164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 100/206] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:18.467303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:18.530669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:18.586480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fce8a-bcc6-4743-8e2a-8d599c3dfcb9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:18.628861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:18.629712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:18.629994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:18.630268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:18.631473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:18.632937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c066412-7b9b-4cdd-a8ee-801e9d0cc981 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:18.634851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 107/207] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:18 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:18.640732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:18.687947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:18.771486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc7ed6-e2ea-4ecb-ade5-a5496c352d57 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:18.809116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:18.809705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:18.809989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:18.810263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:18.811076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b, user.id=13f4e3bea1e041c9bdd3ac55d9589c5b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:18.811385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:18.812851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604b4ef-c608-4424-b3f7-f629f38198d7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:18.819549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 101/208] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:18 2022] GET /identity/v3/role_assignments?user.id=13f4e3bea1e041c9bdd3ac55d9589c5b&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=3d10e61b117d4ae5986d910045db8bea => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:19.079515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3b2c1-fb6b-4b8a-bbde-7796b97273f6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.079698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3b2c1-fb6b-4b8a-bbde-7796b97273f6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.079975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3b2c1-fb6b-4b8a-bbde-7796b97273f6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.081656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 108/209] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:19.087597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.087865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.088137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.090752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f73abd-e57d-4029-8fba-8f938ca7f7cd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.091039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f73abd-e57d-4029-8fba-8f938ca7f7cd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.091314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f73abd-e57d-4029-8fba-8f938ca7f7cd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.092637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 109/210] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:19.098138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.098424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.098695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.123442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:19.124658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:19.125755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:19.127192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:19.217853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 110/211] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:19.226905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 102/212] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:19.242567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.253078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.282865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.302903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.351203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0a118-92aa-4cdd-866b-97d4f84b2c9f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.353544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e84905-d72c-4c0b-871d-fcb34645c9a2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.387483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.388136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.388415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.388686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.389865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.389991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:19.390684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.391058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.391323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:19.391518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.397633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 111/213] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:19.398801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.403317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:19.408504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:19.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:19.410590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:19.411810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 103/214] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:19.417501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.506917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aedb-9641-4247-862a-3eb95da71750 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.513834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.526729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5947a-3c1e-45bc-8da4-53b28afc41a6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.532479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.544321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.545062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.545342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.545646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.546341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:19.547794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa3b8e-87d5-4634-9bf7-13f14784679b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:19.566317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.567028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.567331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.567610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.568521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:19.568872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:19.570357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:19.570729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 112/215] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:19 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:19.574970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bed0-1000-4f56-9e8d-9ff210af660d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.576173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bed0-1000-4f56-9e8d-9ff210af660d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.576452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bed0-1000-4f56-9e8d-9ff210af660d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.581751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 113/216] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:19.585366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 104/217] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:19.591490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.591682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.591906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.592209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.612813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:19.614028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:19.699689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 114/218] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:19.718436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.727079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af1e3-35c5-4ad1-b4a2-d5ca2f93a00e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.779732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.780449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.780731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.781049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] PATH_INFO: `/v3/users/231ebe1ebce7465f9b4ac51bf030b676` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.799353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] RBAC: Authorizing `identity:get_user(user_id=231ebe1ebce7465f9b4ac51bf030b676)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:19.800934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:19.802961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 105/219] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/users/231ebe1ebce7465f9b4ac51bf030b676 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:19.808278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.823514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.840917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb3968-532c-4a42-af44-d8fb7086b91d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.854123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:19.895909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.896492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.896768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.897037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:19.911230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:19.917968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:19.918864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:19.919650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:19.922048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 115/220] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:19.930630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:19.949672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5274-66dd-4fe7-b440-5e747efaecca None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:19.995722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:19.996626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:19.996821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:19.997118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.062743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25eb77-f04d-4a37-90bc-79db70140b29 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.093213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.095987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.102933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.103824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.104309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.104762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.105957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:20.106660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.109336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:20.111227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.123504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 116/221] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:20.132063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:20.171456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:20.257872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 106/222] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:20.264022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:20.271192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30161c4-41c2-411b-aa14-a1af680d9cfa None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.342335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.342988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.343272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.343547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.345249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.346818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.348474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 117/223] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:20 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:20.354119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:20.429131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ceaf4-0f61-4343-b301-67011f0c7b7c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.453788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:20.503208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.504112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.504532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.504943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.506276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.509089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.511608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 107/224] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:20 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:20.518612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:20.545902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:20.551537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd508-e6db-4462-acc1-c5d38400ae9d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.604530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.605189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.605465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.606886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.608560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.610518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.612761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 118/225] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:20 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:20.623334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:20.705540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88079c-d1a9-4714-bf99-3ee3f0ae87f3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.751771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.752417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.752695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.752965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.753792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=231ebe1ebce7465f9b4ac51bf030b676) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:20.754103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.755561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6cc79-cb0b-418f-83bb-812af1ef4cb6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.764376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 108/226] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:20 2022] GET /identity/v3/role_assignments?user.id=231ebe1ebce7465f9b4ac51bf030b676&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:20.823322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e3a5-4d57-4af8-a92f-ce77a493e377 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.823697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e3a5-4d57-4af8-a92f-ce77a493e377 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.823857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e3a5-4d57-4af8-a92f-ce77a493e377 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.825266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 109/227] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:20.830685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96b31f-d27c-4528-96f0-761568b46fd1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:20.838888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.839201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.839433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.870312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:20.871552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:20.917474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:20.918203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:20.918545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:20.918821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/7bcc153e9e8747149d6e87313c554dca/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:20.923811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, user_id=7bcc153e9e8747149d6e87313c554dca, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:20.926751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:20.949974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:20.962322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 119/228] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:20 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/7bcc153e9e8747149d6e87313c554dca/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:21.027544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 110/229] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:21.048687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:21.097717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:21.199214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17daf6f-95a5-49c5-ac4c-6b5faa0c36c8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:21.243432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:21.244450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:21.244956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:21.245415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:21.256557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:21.262315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:21.263200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:21.263996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:21.271266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 120/230] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:21.272873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:21.376282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d4d7d-16bd-421a-89c9-b22971b08712 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:21.412097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:21.413103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:21.413586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:21.414054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:21.414969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:21.415491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:21.417131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:21.426192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 111/231] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:21.432227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:21.546574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:21.568404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9033b-2f7c-489a-a5d7-0e4cf01c7b05 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:21.609011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:21.609793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:21.610076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:21.610376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:21.614277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:21.615754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:21.617413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 121/232] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:21 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:21.623396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:21.662000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:21.732804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452bbb2-402f-4877-8604-1e9cbf766991 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:21.767214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:21.768248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:21.768760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:21.769216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:21.770625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:21.772410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39aece4-84c7-4083-88ef-940f4fd7723d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:21.774343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 112/233] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:21 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:21.780523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:21.905075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73418d-5f56-4697-8eda-723194e6299b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:21.953248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:21.953970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:21.954283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:21.954558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/7bcc153e9e8747149d6e87313c554dca/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:21.962303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, user_id=7bcc153e9e8747149d6e87313c554dca, role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:21.963902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8122d-05a4-41ac-aa7c-12f7ca358f10 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:21.987430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 122/234] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:21 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/7bcc153e9e8747149d6e87313c554dca/roles/0faca02de5824306b70c798c90d5f033 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:22.205490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb7f64-3b05-48a7-a203-3d141a11864c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.206183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb7f64-3b05-48a7-a203-3d141a11864c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.206664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb7f64-3b05-48a7-a203-3d141a11864c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.207139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733a1c-3db3-4f14-a99e-b85ae69aacfb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.207426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733a1c-3db3-4f14-a99e-b85ae69aacfb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.207696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733a1c-3db3-4f14-a99e-b85ae69aacfb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.208860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 113/235] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:22.209536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 123/236] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:22.214981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.215140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.215336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.215536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.215973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.216457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.237222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:22.238444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:22.239183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:22.240659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:22.248511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:22.308338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 124/237] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:22.318005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 114/238] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:22.350905 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.362354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.446066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:22.454539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd04738-a3c8-4578-bb08-59acba113be7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:22.489455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:22.490056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.490333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.490603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.506502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:22.507706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f336-7fd6-48ed-b1ff-882cf63aeda7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:22.511531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:22.512571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:22.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 125/239] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:22.518573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.528336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:22.590287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:22.590966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.591245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.591520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.592773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:22.594641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:22.596289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 115/240] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:22.611027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.630643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314cb9-a96e-4796-854a-59accc69ce8f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:22.667430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:22.668057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.668335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.668597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.669424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:22.669760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:22.671262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:22.680361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 126/241] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:22.685942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.757693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:22.782455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:22.785339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855da89-f671-4bbf-a3b8-e38f78b6bdc5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:22.798592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c90c6-9cee-431d-907a-bf60cc4434e0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:22.822875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:22.823604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.823887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.824154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.825106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:22.826495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:22.836741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:22.837829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:22.838302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:22.838860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] PATH_INFO: `/v3/users/3dde71314da141f2ad29bd62764ee0c9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:22.854180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] RBAC: Authorizing `identity:get_user(user_id=3dde71314da141f2ad29bd62764ee0c9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:22.856010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:22.857989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 127/242] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:22 2022] GET /identity/v3/users/3dde71314da141f2ad29bd62764ee0c9 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:22.863699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:22.863831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 116/243] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:22 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:22.982687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353df6-a19f-424d-aa71-9896f59b86c4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:23.015193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:23.050366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:23.051450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.051919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.052398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.134909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:23.136412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283cba8-2a0a-4b3c-92fa-3604b7afe1e2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:23.242826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 128/244] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:23.248751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:23.360260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3505-578a-4082-b73c-7d217b530e35 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.360617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3505-578a-4082-b73c-7d217b530e35 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.360862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3505-578a-4082-b73c-7d217b530e35 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.362265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 129/245] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:23.367331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.367615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.367887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.387834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:23.389037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:23.417129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:23.480695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7880565-4c58-4b40-bb7a-c3e374443adf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:23.539314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 130/246] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:23.549257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:23.550470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.550773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.552244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.553930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:23.555738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:23.558571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 117/247] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:23 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:23.564970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:23.575446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:23.613296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:23.736092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302896ba-cff0-4bbc-ad9c-a247c8cd8c0b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:23.745092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7291-8551-4ff0-8b51-db553733d323 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:23.781940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:23.783095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.784046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.784511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.787132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:23.787767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.788189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.788590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.789523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=3dde71314da141f2ad29bd62764ee0c9) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:23.790011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:23.791618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:23.798046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 131/248] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:23 2022] GET /identity/v3/role_assignments?user.id=3dde71314da141f2ad29bd62764ee0c9&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=3d10e61b117d4ae5986d910045db8bea => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:23.802698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9fbf5cab-ed39-4fff-b692-b90b3fe3cc7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:23.808795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:23.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:23.810707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:23.813650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 118/249] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:23.821608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:23.884645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:23.885180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8682a2-6c87-4f35-afe1-302dcb306a0d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.886120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8682a2-6c87-4f35-afe1-302dcb306a0d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.886717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8682a2-6c87-4f35-afe1-302dcb306a0d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.890927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 119/250] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:23.900126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239b904-1608-4e41-8c35-8796096878bf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:23.900721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239b904-1608-4e41-8c35-8796096878bf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:23.901182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239b904-1608-4e41-8c35-8796096878bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:23.939563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8239b904-1608-4e41-8c35-8796096878bf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:23.941338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8239b904-1608-4e41-8c35-8796096878bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:24.066253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 120/251] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:24.117149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239b904-1608-4e41-8c35-8796096878bf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.119977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3542ff2-4e40-409d-8f1b-121debde1b24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.137505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:24.222286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.222935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.223211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.223480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.224223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:24.224523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.225950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.230496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239b904-1608-4e41-8c35-8796096878bf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.245094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 132/252] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:24.253859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.284136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.284799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.287106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.287395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.310609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:24.322892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:24.323792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:24.324577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 121/253] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:24.331648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.398042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:24.418749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03c88-e8d7-447a-a1fd-0d36a9218bcd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.461767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.462426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.462853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.463121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] PATH_INFO: `/v3/users/231ebe1ebce7465f9b4ac51bf030b676` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.464459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] RBAC: Authorizing `identity:get_user(user_id=231ebe1ebce7465f9b4ac51bf030b676)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.466518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.468151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 133/254] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:24 2022] GET /identity/v3/users/231ebe1ebce7465f9b4ac51bf030b676 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:24.474365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.515478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:24.579148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518ae71-71b8-49db-ab42-8fbc7003bafc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.586867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2768770-098b-4dd1-9d1a-71fe9208904a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.626053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.626886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.627275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.627639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.629084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.631277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.633078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 134/255] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:24 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:24.635171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.635953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.636352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.636765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.637661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:24.638164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.639164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.640046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.649641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 122/256] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:24.658611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:24.660103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:24.754831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882e93-0198-47a8-a9c0-399e39d057d5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.791952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.792850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.793209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.793545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/231ebe1ebce7465f9b4ac51bf030b676/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.795802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, user_id=231ebe1ebce7465f9b4ac51bf030b676, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.798138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24673c5d-e0d8-4740-a4bd-63fbc754183b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.815719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 135/257] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:24 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/231ebe1ebce7465f9b4ac51bf030b676/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:29:24.817217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dc99c-74cb-4ad3-9149-d98c86de3b43 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.817579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dc99c-74cb-4ad3-9149-d98c86de3b43 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.822707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dc99c-74cb-4ad3-9149-d98c86de3b43 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.824901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 136/258] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:24.831601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.831965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.832299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.866733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:24.868038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:24.899854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c115d8-2488-4a7c-bfef-b3bca881a5a6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:24.991388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:24.992410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:24.992860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:24.993299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:24.994718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:24.996412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:24.998357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 123/259] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:24 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:25.013409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.020262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 137/260] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:25.027063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.061988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.073409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.151748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c3780-5bc1-4d1d-a4d4-ae6599292466 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.191142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.192096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.192381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.192786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.269889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:25.271642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:25.274880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.299563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df366e3e-3722-49e4-a602-9ea7ab889701 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.382908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 124/261] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:25.388981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.426357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.426952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.427232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.427505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.462607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:25.467881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:25.468834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:25.470085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:25.473934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 138/262] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:25.478478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.505262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972a8c-6e67-4e76-8158-742b2b2e8075 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.508030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.547195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.548305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.548774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.549246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.550623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:25.552416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:25.554739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 125/263] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:25.562188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.591264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.626680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c857b57-ae79-47cc-ad87-1579e33d3112 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.660836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.661469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.661789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.662064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.662847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:25.663081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:25.664475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:25.672738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 139/264] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:25.678439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.680200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e3a5-8e55-4875-8f46-5b5b9b13bb4b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.718789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.720228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.720724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.721264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.722398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=7bcc153e9e8747149d6e87313c554dca) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:25.722900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:25.724558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:25.731328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 126/265] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/role_assignments?user.id=7bcc153e9e8747149d6e87313c554dca&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:25.787286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0b9b4-67e0-4b2e-987a-81ebd7e78ba3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:25.816652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.829185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:25.830012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:25.830237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:25.830988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] PATH_INFO: `/v3/users/7bcc153e9e8747149d6e87313c554dca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:25.832861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] RBAC: Authorizing `identity:get_user(user_id=7bcc153e9e8747149d6e87313c554dca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:25.834435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:25.836567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 140/266] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/users/7bcc153e9e8747149d6e87313c554dca => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:25.842580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:25.893516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:25.987185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c8ae8-f088-423e-8079-9788b3804469 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:26.055354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:26.056395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.056876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.057366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.130965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:26.132809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da8608-b24d-4e1e-a751-34f7842c3ddd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:26.168460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:26.240746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 127/267] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:26.250687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:26.362533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c8fd3-efa0-4cfd-acda-a1ea0398928f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.363283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c8fd3-efa0-4cfd-acda-a1ea0398928f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.363851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c8fd3-efa0-4cfd-acda-a1ea0398928f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.369125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 128/268] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:26.372033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.372665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.373159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.379185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202e0a-9042-484d-a35b-37a797ac25ed None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:26.394567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:26.396186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:26.415172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:26.415815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.416091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.416361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.417530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:26.419112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:26.421133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 141/269] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:26.427033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:26.474208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 129/270] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:26.498425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:26.533021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:26.598541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a11da-91b1-4e7b-bf1a-da56bab299d9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:26.603583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:26.664203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a54f-c532-4735-81b8-2530e51d67f6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:26.666289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:26.666925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.667200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.667470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.668273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=7bcc153e9e8747149d6e87313c554dca) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:26.668580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:26.670082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e26a0-8b62-4f41-9587-a030397d6ab1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:26.682662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 142/271] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/role_assignments?user.id=7bcc153e9e8747149d6e87313c554dca&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=0faca02de5824306b70c798c90d5f033 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:26.700462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6298a01-786e-4203-8592-627ac38fdbc3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.700782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6298a01-786e-4203-8592-627ac38fdbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.701008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6298a01-786e-4203-8592-627ac38fdbc3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.702766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 143/272] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:26.708885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc14fc-86b2-46f6-a82e-175fcb1b288c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.709201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc14fc-86b2-46f6-a82e-175fcb1b288c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.709429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc14fc-86b2-46f6-a82e-175fcb1b288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.731222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-18dc14fc-86b2-46f6-a82e-175fcb1b288c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:26.731791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:26.732424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-18dc14fc-86b2-46f6-a82e-175fcb1b288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:26.732524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.732745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.733022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.749858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5080d54f-3ada-4c14-b02e-0837e1e8fb07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:26.756832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:26.757819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:26.758616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:26.759224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 130/273] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:26.760219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766b77-06a6-4738-b365-9ca75f0a93fa None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.760569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766b77-06a6-4738-b365-9ca75f0a93fa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.761108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766b77-06a6-4738-b365-9ca75f0a93fa None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.763041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 131/274] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:26.768440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766b77-06a6-4738-b365-9ca75f0a93fa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:26.814779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 144/275] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:26.822875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.823210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.823435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.857927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:26.859103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:26.882601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766b77-06a6-4738-b365-9ca75f0a93fa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:26.914487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:26.924495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:26.925129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:26.925408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:26.925723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:26.926450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:26.926750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:26.927376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:26.928563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:26.937732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 132/276] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:26.938597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:26.946811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 145/277] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:26.947645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.062509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e520ca-2580-4806-a163-0ee7762ac754 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.066044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d04d2-7eee-4d95-9bca-af87f84489ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.099935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.100275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.100592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.100873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.100988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.101141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] PATH_INFO: `/v3/users/8e6504b9ba84436d80b6b3b3a5e59151` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.101231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.101415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.112200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:27.115002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] RBAC: Authorizing `identity:get_user(user_id=8e6504b9ba84436d80b6b3b3a5e59151)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.116479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.118361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 146/278] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/users/8e6504b9ba84436d80b6b3b3a5e59151 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:27.118463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:27.119282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:27.120065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 133/279] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:27.123851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.126340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.177138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.178085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.228756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c754a-0e67-4611-a181-116c487af130 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.237477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaf9a0-bd22-45f3-abe9-8645ac1206d8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.264351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.264878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.265174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.265448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.266269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, user.id=8e6504b9ba84436d80b6b3b3a5e59151, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:27.266606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.268090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.281582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 134/280] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/role_assignments?user.id=8e6504b9ba84436d80b6b3b3a5e59151&scope.system=all&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:27.281582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.281880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.281880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.281880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.281880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.282141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.282141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.282141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 147/281] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:27.286372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.405597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1aff9-c89e-4122-be7a-2d81611cb33a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.413818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b77db-9cde-466e-942a-e7922fb1b174 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.421478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.445779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.449951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.450828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.451296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.451754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.452705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:27.453181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.455569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.463277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.463916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.464197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.464472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.465165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.466155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 135/282] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:27.469601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.472528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.496856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 148/283] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:27 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:27.601236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab402f3-ad13-4f3b-8729-1ed1a5d700ad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.641888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.643065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.643577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.644045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] PATH_INFO: `/v3/users/3dde71314da141f2ad29bd62764ee0c9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.645672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] RBAC: Authorizing `identity:get_user(user_id=3dde71314da141f2ad29bd62764ee0c9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.647440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecabd6-405e-40e3-a22b-30b4de80c38e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.649373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 136/284] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/users/3dde71314da141f2ad29bd62764ee0c9 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:27.655444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.674439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f56215-340c-4837-87b5-d9bdec7ec341 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.674904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f56215-340c-4837-87b5-d9bdec7ec341 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.675189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f56215-340c-4837-87b5-d9bdec7ec341 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.676993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 137/285] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:27.682264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.682545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.682820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.703757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:27.705424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:27.750131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.793185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a805-d2b0-4669-a675-41c56787ba4b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.796046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 138/286] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:27.819997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.836387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.837053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.837330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.837628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:27.838818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:27.840267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:27.842335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 149/287] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:27.851233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:27.904739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:27.953083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd8ee7-6bca-4209-bc57-629c8562f75f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.983221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f19b48-944c-4c19-8855-9b8e7a72da53 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:27.994442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:27.995425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:27.996003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:27.996490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.009341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:28.014968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:28.015898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:28.016753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:28.020153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 139/288] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:28.020982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:28.026926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:28.026926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:28.026926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/3dde71314da141f2ad29bd62764ee0c9/roles/3d10e61b117d4ae5986d910045db8bea` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.029339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, user_id=3dde71314da141f2ad29bd62764ee0c9, role_id=3d10e61b117d4ae5986d910045db8bea)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:28.029716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:28.031904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:28.050973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 150/289] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:29:27 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/users/3dde71314da141f2ad29bd62764ee0c9/roles/3d10e61b117d4ae5986d910045db8bea => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:29:28.068099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:28.154958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fdc2d-6f71-4641-97db-f4492d0b4544 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:28.198220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:28.199288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:28.199750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:28.200162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.201039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:28.201479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:28.203121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:28.217600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 140/290] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:28.218673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:28.361098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:28.365165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24558e1-f54d-4d8f-8069-e3eeb5bd8afe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:28.399590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:28.400244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:28.400523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:28.400803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] PATH_INFO: `/v3/users/231ebe1ebce7465f9b4ac51bf030b676` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.402022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] RBAC: Authorizing `identity:get_user(user_id=231ebe1ebce7465f9b4ac51bf030b676)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:28.403462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:28.405632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 151/291] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:28 2022] GET /identity/v3/users/231ebe1ebce7465f9b4ac51bf030b676 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:28.410935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:28.531151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:28.538433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac628a2-c4fa-434d-bf62-9fea22ef8b4c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:28.582218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:28.582854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:28.583138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:28.583412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.673212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:28.674649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:28.786823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:28.799188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 141/292] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:28.800600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:28.913599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc043b0-7c75-4935-8848-b39307a375a3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:28.949693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:28.950265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:28.950548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:28.950826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:28.951997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:28.953441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f56022-16db-4495-b2a1-c50a309e230c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:28.955191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 152/293] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:28 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:28.961462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:29.086846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28403865-f0fc-4617-adb8-00dbc133c5a4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:29.134206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:29.134881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:29.135164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:29.135490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:29.136358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=231ebe1ebce7465f9b4ac51bf030b676) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:29.136671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:29.138146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc5db5-ebba-48dc-a8a4-40283240a846 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:29.150211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 142/294] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:28 2022] GET /identity/v3/role_assignments?user.id=231ebe1ebce7465f9b4ac51bf030b676&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:29.548856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddaff4-70b8-4fba-a907-14e5ed12a78e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:29.549220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddaff4-70b8-4fba-a907-14e5ed12a78e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:29.549410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddaff4-70b8-4fba-a907-14e5ed12a78e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:29.550893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 153/295] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:29.556442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:29.556738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:29.557019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:29.577789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:29.578974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:29.648147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:29.669599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 143/296] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:29.694158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddaff4-70b8-4fba-a907-14e5ed12a78e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:29.745969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:29.808731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddaff4-70b8-4fba-a907-14e5ed12a78e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:29.843109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:29.843761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:29.844041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:29.844713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:29.855321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:29.860843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:29.861748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:29.862555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 154/297] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:29.868294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:29.971034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f47ad0-4e50-48cb-a9b8-01ec23d86f02 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.016534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.017293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.017590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.017872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.018606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:30.018909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.020314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.029617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 144/298] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:30.035153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.134061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.139906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e65c1-b671-4aa1-acc4-ebe8f1bf5595 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.175120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.175837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.176122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.176398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] PATH_INFO: `/v3/users/8e6504b9ba84436d80b6b3b3a5e59151` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.177764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] RBAC: Authorizing `identity:get_user(user_id=8e6504b9ba84436d80b6b3b3a5e59151)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.179232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b73f-77df-4836-b20b-66e5127a8e24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.185468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 155/299] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/users/8e6504b9ba84436d80b6b3b3a5e59151 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:30.187047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.230693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07865f95-3f8c-4c8f-877c-3538ae03f2bd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.231043 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07865f95-3f8c-4c8f-877c-3538ae03f2bd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.231247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07865f95-3f8c-4c8f-877c-3538ae03f2bd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.232971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 156/300] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:30.237585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.237879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.238207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.243790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.259924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:30.261120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:30.310297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc55e-2a2f-46be-8954-b30ad888f545 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.340704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 157/301] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:30.351705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.352789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.353076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.353354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] PATH_INFO: `/v3/system/users/8e6504b9ba84436d80b6b3b3a5e59151/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.354914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8e6504b9ba84436d80b6b3b3a5e59151, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.356324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5babe98-d95a-4ab0-8aac-3c711a6a429a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.359774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.366841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 145/302] 10.222.0.47 () {58 vars in 1334 bytes} [Mon Aug 1 22:29:30 2022] PUT /identity/v3/system/users/8e6504b9ba84436d80b6b3b3a5e59151/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:30.382980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9009-2a51-49df-9b6e-b4443c1434be None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.383360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9009-2a51-49df-9b6e-b4443c1434be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.383646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9009-2a51-49df-9b6e-b4443c1434be None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.385752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 146/303] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:30.391058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb12a-fede-43eb-9243-a4ceec2ef54f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.391406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb12a-fede-43eb-9243-a4ceec2ef54f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.391673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb12a-fede-43eb-9243-a4ceec2ef54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.412568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64ceb12a-fede-43eb-9243-a4ceec2ef54f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:30.414300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64ceb12a-fede-43eb-9243-a4ceec2ef54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:30.464426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.474720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4f2af-9f22-4eb2-827c-e6772cbe3d15 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.493014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 147/304] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:30.494396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496da7d0-5505-421d-857f-230e056cef71 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.494736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496da7d0-5505-421d-857f-230e056cef71 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.495020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496da7d0-5505-421d-857f-230e056cef71 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.496537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 148/305] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:30.502142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.502454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.502694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.521698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.522368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.522705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.522988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.523590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:30.524808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:30.538302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:30.543408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:30.544331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:30.545113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 158/306] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:30.546370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.570318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.602218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 149/307] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:30.606514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.663487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f76c0-df6b-485b-a2e8-51c9ce5638d6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.700770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.701427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.701737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.702002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.711165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0639fa-d5a8-4a09-88f1-3b65ccbb30b3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.712646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:30.717675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:30.718551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:30.719734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 159/308] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:30.720425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.741404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.752821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.753457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.753803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.754041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.754774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:30.755136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.756591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.765673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 150/309] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:30.765942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.802484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:30.840759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e05a-c1d7-4b36-afe1-2d2c9a106365 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.872970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf44b2-2be5-4ff2-9858-1a081af1d7cb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:30.875697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.876389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.876684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.876960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.878143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.879613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.881231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 160/310] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:30.886582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.911469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:30.912370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:30.912828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:30.913278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:30.914276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:30.914775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:30.916337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:30.925353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 151/311] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:30.926542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:30.977668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.002637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6bb6-fc7b-444c-88e9-19ccdbaabb1a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.040908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.043061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.043782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.044022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.044292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] PATH_INFO: `/v3/users/6b46cea8ee034478a36390d9c32684c9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.046224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22429a4c-98b5-440f-be61-39ddb2e1e810 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.058296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=6b46cea8ee034478a36390d9c32684c9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.059896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.061634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 161/312] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:30 2022] GET /identity/v3/users/6b46cea8ee034478a36390d9c32684c9 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:31.066520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:31.086136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.086717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.086992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.087277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] PATH_INFO: `/v3/groups` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.087923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.089367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.117588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 152/313] 10.222.0.47 () {60 vars in 1134 bytes} [Mon Aug 1 22:29:30 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:31.117588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:31.177810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83f9ca-c771-42e8-8021-6f35f9b53ba3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.216979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.217666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.217943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.218214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] PATH_INFO: `/v3/users/3dde71314da141f2ad29bd62764ee0c9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.220107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] RBAC: Authorizing `identity:get_user(user_id=3dde71314da141f2ad29bd62764ee0c9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.222684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.224498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 162/314] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:31 2022] GET /identity/v3/users/3dde71314da141f2ad29bd62764ee0c9 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:31.230630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:31.241078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.265295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bece65f-91dc-4fa9-8604-f6c6c84b45bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.308609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.309245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.309523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.309825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.311180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, user.id=6b46cea8ee034478a36390d9c32684c9, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:31.311180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.312347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.319527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 153/315] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Aug 1 22:29:31 2022] GET /identity/v3/role_assignments?user.id=6b46cea8ee034478a36390d9c32684c9&scope.system=all&role.id=3d10e61b117d4ae5986d910045db8bea => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:31.364465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252d88c-14d3-4221-9795-edb339e139d5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.403783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.404419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.404691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.405025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.486393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.488294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.497890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.595491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 163/316] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:31.601148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:31.628598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.722777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5f47-c1c2-4f84-8b6b-eb2b7beb69ab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.757948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.758955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.759431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.759894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.761253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.762949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968df980-231e-4ccc-b6d6-2059aaffb521 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.765143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 154/317] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:31 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:31.771216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:31.876269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577c54-3cdf-43ae-a6cf-1bbc4de06be9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:31.884057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:31.912745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:31.913373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:31.913682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:31.913956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:31.914759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a, user.id=3dde71314da141f2ad29bd62764ee0c9) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:31.915066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:31.916586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05a02-9610-4a6f-8357-109cfdf864c1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:31.923152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 164/318] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:31 2022] GET /identity/v3/role_assignments?user.id=3dde71314da141f2ad29bd62764ee0c9&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=3d10e61b117d4ae5986d910045db8bea => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:32.394537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718c945-1902-442e-a224-8fa87420cfce None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:32.395050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718c945-1902-442e-a224-8fa87420cfce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:32.395050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718c945-1902-442e-a224-8fa87420cfce None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:32.396503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 155/319] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:32.401589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:32.401990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:32.402172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:32.420499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:32.421731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:32.494594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 165/320] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:32.517687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718c945-1902-442e-a224-8fa87420cfce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:32.610942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:32.622962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718c945-1902-442e-a224-8fa87420cfce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:32.656089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:32.656690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:32.656965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:32.657255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:32.667461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:32.672357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:32.673303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:32.674486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:32.677871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 156/321] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:32.679582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:32.780557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:32.784628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2cfe-15af-4170-a86e-a5d380cda57c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:32.821015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:32.821671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:32.821929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:32.822193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:32.822912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:32.823207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:32.824576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:32.837474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 166/322] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:32.839172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:32.945155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b7359-955f-4b64-8059-ce55534b70e6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:32.980332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:32.980981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:32.981262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:32.981536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] PATH_INFO: `/v3/users/8e6504b9ba84436d80b6b3b3a5e59151` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:32.982745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] RBAC: Authorizing `identity:get_user(user_id=8e6504b9ba84436d80b6b3b3a5e59151)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:32.984214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e014e-e314-45d4-9416-ce2b6bc358e3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:32.986104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 157/323] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:32 2022] GET /identity/v3/users/8e6504b9ba84436d80b6b3b3a5e59151 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:32.992031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.099599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ccce8-0ec7-434a-a5d2-fa0d90b59227 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.134232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.134856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.135132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.135401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.136188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] RBAC: Adding query filter params (role.id=d9a82ca5f4b244f49e8a8ba0d7999547, user.id=8e6504b9ba84436d80b6b3b3a5e59151, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:33.136483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:33.137969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a639cf-3806-44fc-8764-6960d2af37c0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:33.152153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 167/324] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Aug 1 22:29:32 2022] GET /identity/v3/role_assignments?user.id=8e6504b9ba84436d80b6b3b3a5e59151&scope.system=all&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:33.247056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416499de-878f-4ab6-a4b5-4fbc1a7ed79f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.247311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416499de-878f-4ab6-a4b5-4fbc1a7ed79f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.247570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416499de-878f-4ab6-a4b5-4fbc1a7ed79f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.249159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 158/325] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:33.254819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.255113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.255364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.274723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:33.275918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:33.291597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:33.349455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 168/326] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:33.371652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.372089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225d3fe-0164-442b-9e95-d33177ea8227 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.372370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225d3fe-0164-442b-9e95-d33177ea8227 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.372639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225d3fe-0164-442b-9e95-d33177ea8227 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.374029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 159/327] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:33.380542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.380935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.381331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.399662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:33.400866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:33.414113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:33.471160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 160/328] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:33.472998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a120a3a-e200-4e62-aa25-143fcddd1849 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.492786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.509124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.509738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.510017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.510290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.511464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:33.512900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:33.514540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 169/329] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:33.520270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.538537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:33.589711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef126c4-dce8-4caf-a936-7e988c014cfc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.620566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229548a-84cf-4017-bc79-e8addd363f38 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.623384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.624029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.624305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.624571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.634401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:33.639528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:33.640473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:33.641417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:33.646875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 161/330] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:33.647295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.654696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.655333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.655618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.655894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] PATH_INFO: `/v3/groups` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.656566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:33.658129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:33.660740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:33.676263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 170/331] 10.222.0.47 () {60 vars in 1134 bytes} [Mon Aug 1 22:29:33 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:29:33.764435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe8c5-f20b-4644-8cdd-11ef5e00badc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.803747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.804382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.804660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.804935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.805677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:33.805984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:33.807432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:33.817046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 162/332] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:33.822735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:33.904538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:33.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436578-e786-439b-8c55-3803f26ed387 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:33.966091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:33.966773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:33.967047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:33.967324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] PATH_INFO: `/v3/users/6b46cea8ee034478a36390d9c32684c9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:33.968494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] RBAC: Authorizing `identity:get_user(user_id=6b46cea8ee034478a36390d9c32684c9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:33.970369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7a648-6249-400c-b4a9-cd59eb32d227 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:33.972155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 171/333] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:33 2022] GET /identity/v3/users/6b46cea8ee034478a36390d9c32684c9 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:33.977835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:34.064122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:34.097529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8ca2c-7438-4c7c-b2c9-64b524dc13c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:34.132755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:34.133437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:34.133740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:34.134018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] PATH_INFO: `/v3/system/users/6b46cea8ee034478a36390d9c32684c9/roles/3d10e61b117d4ae5986d910045db8bea` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:34.135978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6b46cea8ee034478a36390d9c32684c9, role_id=3d10e61b117d4ae5986d910045db8bea)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:34.137955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f9bc9-43ea-452e-a669-efa24523dc6d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:34.148523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 163/334] 10.222.0.47 () {58 vars in 1334 bytes} [Mon Aug 1 22:29:33 2022] PUT /identity/v3/system/users/6b46cea8ee034478a36390d9c32684c9/roles/3d10e61b117d4ae5986d910045db8bea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:36.031594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811075b-bb0c-4e76-b78b-671bdd3dd8aa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.032110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811075b-bb0c-4e76-b78b-671bdd3dd8aa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.032224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811075b-bb0c-4e76-b78b-671bdd3dd8aa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.033543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 172/335] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:36.039266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.039700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.039867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.061584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:36.062941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:36.148146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 164/336] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:36.197020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811075b-bb0c-4e76-b78b-671bdd3dd8aa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:36.319142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811075b-bb0c-4e76-b78b-671bdd3dd8aa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:36.365045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:36.365746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.366025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.366299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.377432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ed0aa8aa-8b71-4393-9715-26fe5058d54e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:36.385213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:36.386196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:36.387142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 173/337] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:36.393837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:36.479476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:36.527568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b25b5-14cf-4087-b473-d8d5c9655d26 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:36.544919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67082cc-9620-46d1-81cb-5ff7416a6933 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.545540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67082cc-9620-46d1-81cb-5ff7416a6933 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.546073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67082cc-9620-46d1-81cb-5ff7416a6933 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.547937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 174/338] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:36.555009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.555599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.556062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.579429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:36.579982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.580260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.580527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.581254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:36.581578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:36.589579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:36.592034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 165/339] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:36.597474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:36.602873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:36.605126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:36.613290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:36.756780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 175/340] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:36.785388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:36.822992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd74b-39bf-4f65-b8b7-431dc0a8f6df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:36.855450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:36.869139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:36.870309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.871568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.872211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.943840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362220f0-7002-43aa-88b0-4bd7df8b399e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:36.982831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:36.983488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:36.983765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:36.984034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:36.986894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:36.995099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fad1ac14-4e5c-4f6a-9324-bde7c0e115a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:37.003694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:37.004831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:37.005870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:37.008752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 176/341] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:37.010418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:37.013135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631fbc-2ed2-4a33-b701-f7284b7b338f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.013419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631fbc-2ed2-4a33-b701-f7284b7b338f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.013722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631fbc-2ed2-4a33-b701-f7284b7b338f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.015119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 177/342] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:37.015442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514c4aa-2d47-443d-92da-5e18fa0776c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:37.016150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84ab4-1a46-4b8b-92c3-d54925943262 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.016416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84ab4-1a46-4b8b-92c3-d54925943262 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.016731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84ab4-1a46-4b8b-92c3-d54925943262 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.017928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 178/343] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:37.021683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e84ab4-1a46-4b8b-92c3-d54925943262 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.137802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 166/344] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:37.139119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.139411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.139683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.148765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e84ab4-1a46-4b8b-92c3-d54925943262 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.164189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:37.165488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:37.190029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.194343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.195327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.196779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.196779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.197374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:37.198149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:37.200635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc3c6b-fa96-4cf1-9121-e474088d8f12 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:37.211582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 179/345] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:37.213093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.214538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.214994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.238819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:37.240753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:37.248698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 167/346] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:37.250787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.318432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.337945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 180/347] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:37.339955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.373030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38564f-1887-41fc-8f2f-f6308bc4939a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.415026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8159c8-3512-438d-b170-293616172996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.415734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8159c8-3512-438d-b170-293616172996 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.416082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8159c8-3512-438d-b170-293616172996 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.416473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8159c8-3512-438d-b170-293616172996 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.419125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8159c8-3512-438d-b170-293616172996 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:37.420679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8159c8-3512-438d-b170-293616172996 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:37.422919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 168/348] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:37.423829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8159c8-3512-438d-b170-293616172996 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.465412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.475086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d716-9705-463d-816c-640bfd4ec52b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.523971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.524961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.525715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.526203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.567736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8159c8-3512-438d-b170-293616172996 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.599762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.613944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.614977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.615700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.616166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.618888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:37.620470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:37.630840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:37.638174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:37.639193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:37.640020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:37.642772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 169/349] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:37.643818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.763928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 181/350] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:37.765601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.765729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.789710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4d4eb-f372-408c-b985-a0e7a8b0faa4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.839689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.840354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.840639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.840912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.854360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:37.859915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:37.861164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:37.862128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 170/351] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:37.863037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:37.896112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:37.916500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1091d-c6de-484d-867f-0a51e2f327f4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:37.959615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:37.960263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.960544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.960815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.968039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:37.970154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132c3d-5996-443b-85d2-31cfb3def39d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:37.972884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 182/352] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:37.974522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af2b-aa57-4c8f-a2d0-6004960d6428 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:37.974875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af2b-aa57-4c8f-a2d0-6004960d6428 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:37.975215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741af2b-aa57-4c8f-a2d0-6004960d6428 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:37.977236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 183/353] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:37.978073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741af2b-aa57-4c8f-a2d0-6004960d6428 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.019216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6bfcc-b85e-4d08-b497-773dc79eec96 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.056955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.057611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.057893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.058164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.058923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:38.059224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.060609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.069254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 171/354] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.070122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.080019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.105106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741af2b-aa57-4c8f-a2d0-6004960d6428 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.153404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.154196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.154433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.154713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.155897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.157381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.159154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 184/355] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:37 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.159998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.186002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.194616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b325e26-6a9a-4c34-aaba-735e1f32db41 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.239403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.240058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.240336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.240612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.241353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:38.241695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.243111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc060-a745-4154-88d6-4d0a58c4828b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.253544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 172/356] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.254965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.255528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.255815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.277805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:38.278949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:38.287821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b98e-b22a-4abb-b4d0-9b4dd4823bee None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.325785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.326544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.326687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.326960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.327767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:38.328071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.329716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1de75-3791-4c80-9d2c-4a704ee4ee79 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.336874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 185/357] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.338212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fa3a-4b77-4f28-9df5-e9770a2c2144 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.338499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fa3a-4b77-4f28-9df5-e9770a2c2144 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.338804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fa3a-4b77-4f28-9df5-e9770a2c2144 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.340303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 186/358] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:38.341157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fa3a-4b77-4f28-9df5-e9770a2c2144 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.353887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.364311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 173/359] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:38.366126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.456107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.483601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fa3a-4b77-4f28-9df5-e9770a2c2144 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.523751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0015-2f99-4e68-af27-f260b315e51c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.537380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.538029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.538318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.538593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.584597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.585747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.585747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.585925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.587899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.588611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.590616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 174/360] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.594738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.628623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.630694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe1e18-11a7-457d-a9cb-7f316ceaaf63 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.697294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.732941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.740450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42457bf-1f15-4e58-83db-d2e8a32d2ae2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.759699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 187/361] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:38.759699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.759699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.759699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.781624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:38.782966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:38.785477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.786438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.786859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.787370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] PATH_INFO: `/v3/users/6b46cea8ee034478a36390d9c32684c9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.788807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] RBAC: Authorizing `identity:get_user(user_id=6b46cea8ee034478a36390d9c32684c9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:38.790557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:38.792543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 175/362] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/users/6b46cea8ee034478a36390d9c32684c9 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:38.793688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.863978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 188/363] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:38.865033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:38.932324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a00a0-7627-4de1-9f40-6b255a6e4aae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.972224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:38.973263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:38.973791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:38.974263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:38.977794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:38.982213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf8ec7-6bb1-4437-aa0a-a175f0729d94 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:38.992963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:38.998658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:38.999568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:39.000369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 176/364] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:39.001074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.002824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.029495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.030430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.030711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.030981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.031781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:39.033069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.035154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.043793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 189/365] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:38 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=0faca02de5824306b70c798c90d5f033 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.044552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.133243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f26a-eafc-48f4-aae9-b403ff90e4f2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.175630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.176295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.176577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.176850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.178085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.179567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.181469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 177/366] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.182049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2998f50-a30f-4b97-bb83-4b808fd35d8e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.184351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.236480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.237125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.237398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.237752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.238513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] RBAC: Adding query filter params (role.id=3d10e61b117d4ae5986d910045db8bea, user.id=6b46cea8ee034478a36390d9c32684c9, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:39.238908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.240247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.257442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 190/367] 10.222.0.47 () {56 vars in 1308 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/role_assignments?user.id=6b46cea8ee034478a36390d9c32684c9&scope.system=all&role.id=3d10e61b117d4ae5986d910045db8bea => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.258338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.268919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.330395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b69b23-5ba6-4600-b345-e878848368ef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.344171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.404285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.404959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.405234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.405501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.418289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:39.427543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:39.428503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:39.429286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 178/368] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:39.430569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.466378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce7b1-d6bd-4923-9f4e-4e32955e8c02 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.512217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.512974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.513253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.513526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.514345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:39.514650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.516172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.529828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 191/369] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.530720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.560023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011f8e-63eb-4ec9-bac6-2a407add88ec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.585598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.601975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.602650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.602977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.603256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.604587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.606114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.608178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 179/370] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.609160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.661413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.666328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e409c-c3ea-4160-b21d-cd1771f7c117 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.701100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.701758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.702035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.702304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.703032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:39.703328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.704768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.713299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 192/371] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.714148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.748393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe22502-e584-4311-9255-3443fc0b959b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.786501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.787164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.787425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.787702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.819932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80169072-0cfc-4c5c-a462-93a4383c3730 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.851101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.858499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:39.859135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:39.859403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:39.859672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:39.860621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:39.860927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.862565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.863891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:39.865502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:39.868986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 193/372] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:39.869821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:39.899185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:39.979836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 180/373] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:39.982230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc711-4d9b-4600-9548-ea1cedd73dcd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:39.986441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.050363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.051387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.051895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.052359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.101672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.107574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2eb18-2904-4a7a-a138-5d38e35df444 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.145037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.145913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.146295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.146674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.147009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.148068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.148828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.149732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.154496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 181/374] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:40.156898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.180823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.265130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 194/375] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:40.267930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368309be-ac0e-4e54-ab07-93cacb522739 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.274645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.311119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.312562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.313142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.313707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.315178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.316952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.319223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 182/376] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:40 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:40.327422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.358187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.404691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2013c8-be3f-4b1d-9cdf-376e1092cc1a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.437593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.452780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.453813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.454387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.455594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] PATH_INFO: `/v3/projects/e92bd71b1dab4dce929247e99aba6067` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.457759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] RBAC: Authorizing `identity:get_project(project_id=e92bd71b1dab4dce929247e99aba6067)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.459604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.462851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 195/377] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:40 2022] GET /identity/v3/projects/e92bd71b1dab4dce929247e99aba6067 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:40.466939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17e812-92ed-4a8b-ab25-3b631c20d94b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.468394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.544548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.545669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.546152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.546646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.547638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:40.548308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.554102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.565441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 183/378] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:40 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=0faca02de5824306b70c798c90d5f033 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:40.610189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7a3-3276-47af-ae67-9080724952ca None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.656353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.657048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.657435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.657757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] PATH_INFO: `/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.664035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] RBAC: Authorizing `identity:get_group(group_id=cc2cfb08f59748f09a88ed3098c58ce0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.665525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5dd3d2-5547-4794-a4d5-f656724e974f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.667680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 196/379] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:40 2022] GET /identity/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:40.673859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:40.809804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.824740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7952-5138-4dda-810b-2eb55b20615c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:40.830250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e272-c18d-43f1-9b05-7cb96c10c39a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.830540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e272-c18d-43f1-9b05-7cb96c10c39a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.830815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e272-c18d-43f1-9b05-7cb96c10c39a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.832446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 197/380] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:40.843986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.844271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.844976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.863677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:40.864284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:40.864558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:40.864826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:40.866020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] RBAC: Adding query filter params (group.id=cc2cfb08f59748f09a88ed3098c58ce0, role.id=c512669c168542d587f922137072be5f, scope.project.id=e92bd71b1dab4dce929247e99aba6067) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:40.866020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:40.867399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:40.875550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 184/381] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:40 2022] GET /identity/v3/role_assignments?group.id=cc2cfb08f59748f09a88ed3098c58ce0&scope.project.id=e92bd71b1dab4dce929247e99aba6067&role.id=c512669c168542d587f922137072be5f => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:40.879646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:40.880907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:40.955253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:40.979275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 198/382] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:41.018087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.160608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c888bc-577b-4754-bea7-e224bd25e56d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:41.203671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:41.204413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.204695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.204970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.216065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:41.225647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:41.226535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:41.227380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 185/383] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:41.232992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.361154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd510-7d43-4286-b157-d0ba82bada7c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:41.404508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:41.405511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.406004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.406459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.407423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:41.407951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:41.409548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3685ecb-14a5-456b-a12b-27f2bdd05a94 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:41.420962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 199/384] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:41.426984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.462689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:41.535384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c60635-f8cb-42b6-b5fd-94c2a55522d8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.536019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c60635-f8cb-42b6-b5fd-94c2a55522d8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.536508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c60635-f8cb-42b6-b5fd-94c2a55522d8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.538086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 200/385] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:41.547655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.548318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.548876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.569678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:41.571174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:41.591050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aae22-e9b3-4dfe-967c-0d2e287eacc3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:41.656664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:41.657390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.657699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.658570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.660243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:41.661737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:41.666214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 186/386] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:41.667614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:41.669224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.717615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 201/387] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:41.752457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.827849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:41.848314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b789-123e-468c-8366-f29bf8dc99fe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:41.885991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:41.886653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.886934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.887207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.888501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:41.890130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:41.892370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 187/388] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:41.902516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:41.915994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce36265-1d88-40ac-93f1-3396f54a18de None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:41.950977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:41.951628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:41.951904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:41.952173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:41.962388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:41.967915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:41.968829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:41.970106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 202/389] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:41.977229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:42.062055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:42.096645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6030b-303d-49cc-b46f-5e7beb8c9385 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:42.154239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504945d3-bc12-4f26-8626-bfe992d81f0c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:42.158077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:42.158767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.159042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.159313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/groups/67da537b29d4404b8b809e3a62301b3d/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.161711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, group_id=67da537b29d4404b8b809e3a62301b3d, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:42.163353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:42.188934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 188/390] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Aug 1 22:29:41 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/groups/67da537b29d4404b8b809e3a62301b3d/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:42.205088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:42.205758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.206024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.206294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.207080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:42.207385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:42.208763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:42.220322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 203/391] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:42.225830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:42.241479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:42.365528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d6a7d-ee14-4b8e-a534-5d7916e37e7d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:42.412237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:42.412899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.413177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.413450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.414619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:42.416078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:42.419195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 189/392] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:42 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:42.425269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:42.528955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:42.532459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609fbf-c27b-457a-afdf-1f656a12dd2b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:42.566939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:42.567594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.567875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.568148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.569315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:42.570806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98e01e-79c4-4af4-9306-f854418dd516 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:42.572779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 204/393] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:42 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:42.578612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:42.623656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc301-ecf7-4aa3-a088-d15beb8324ac None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.624002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc301-ecf7-4aa3-a088-d15beb8324ac None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.624199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc301-ecf7-4aa3-a088-d15beb8324ac None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.625767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 205/394] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:42.633601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcc23-abd3-4992-a58b-84dab298375c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.633601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcc23-abd3-4992-a58b-84dab298375c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.633601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcc23-abd3-4992-a58b-84dab298375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.650956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-edebcc23-abd3-4992-a58b-84dab298375c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:42.652344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-edebcc23-abd3-4992-a58b-84dab298375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:42.681971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:42.698752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98516-805d-489e-9e81-a35d0bbac62c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:42.736010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:42.736953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:42.737710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:42.738146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/groups/67da537b29d4404b8b809e3a62301b3d/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:42.740622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] RBAC: Authorizing `identity:create_grant(project_id=2df66faea5004c4c93c8762a9ae43a4b, group_id=67da537b29d4404b8b809e3a62301b3d, role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:42.742473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:42.762366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 190/395] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Aug 1 22:29:42 2022] PUT /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b/groups/67da537b29d4404b8b809e3a62301b3d/roles/0faca02de5824306b70c798c90d5f033 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:42.804539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 206/396] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:42.841049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:42.963520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e50fa-2a87-45aa-8ca9-3b4eb0f9952a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.006567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.007236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.007512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.007779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.027761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:43.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:43.033856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 191/397] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:43.034866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.042321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245d31-b935-417c-96be-cc50f9f3062a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.042609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245d31-b935-417c-96be-cc50f9f3062a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.042866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245d31-b935-417c-96be-cc50f9f3062a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.044417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 207/398] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:43.045291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.050475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.050814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.050983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.078526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:43.079828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:43.165712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 208/399] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:29:43.179573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d46a9-d16b-4bf5-8d5d-db330738fbd4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.182694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.210933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.222136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.224531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.225735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.226304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.227236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:43.228889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.230525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.249162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 192/400] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:43.255713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.329132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360c71e-29d0-4830-b1b6-7fa731f31b48 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.356155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.387121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.387784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.388056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.388325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.405238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:43.410604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:43.411521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:43.412742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 209/401] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:43.419556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.439375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc2417-3323-4367-9b37-54b2f4402183 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.495486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.496808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.497324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.497835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.499632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.501319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.508242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 193/402] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:43.510199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.581991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.605769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd5ef-8668-4294-abaf-7300b8bb10a9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.620190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0622e7-c7d9-4140-ab42-51fd49be5a61 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.648916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.654196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.654842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.655546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.655831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.656683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:43.657154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.658592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.660558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.661277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.661662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.661997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.663229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.664725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515db2b-cb28-4308-9141-e736d11e1b49 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.666814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 194/403] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:43.667946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 210/404] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:43.668127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377921-ffa2-4735-bde1-211ef4704893 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.668504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377921-ffa2-4735-bde1-211ef4704893 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.668891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377921-ffa2-4735-bde1-211ef4704893 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.670512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 195/405] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:43.678364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.679749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da377921-ffa2-4735-bde1-211ef4704893 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.794458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6be327-6309-4117-83ca-7f6058de98fe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.820297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da377921-ffa2-4735-bde1-211ef4704893 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:43.830257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.831111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.831549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.832064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.833374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.835051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fa7dc-627d-42df-b6f7-69e43d8a4466 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.841527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 211/406] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:43.843081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.843577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.843982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.860170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:43.861308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:43.861712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:43.862091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/groups/67da537b29d4404b8b809e3a62301b3d/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:43.864117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, group_id=67da537b29d4404b8b809e3a62301b3d, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:43.865465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:43.865776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:43.867048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:43.889270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 196/407] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Aug 1 22:29:43 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/groups/67da537b29d4404b8b809e3a62301b3d/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:43.889270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:43.889533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:43.996251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 212/408] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:44.021059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.048995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:44.125450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e07867-8e4b-4fa2-9258-3173ca4da6b2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.136245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919e17a-d16e-4dbb-9dd9-86de0fadba7e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.174636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.175450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.175902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.176315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.176464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.176757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.177031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.177189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.178665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:44.181037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:44.183227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 197/409] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:43 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:44.190380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.197771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:44.208377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:44.209548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:44.210443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:44.212753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 213/410] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:44.223212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.241833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:44.339170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb0d3-bbca-4029-881a-1e07596c619e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.367568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:44.382456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.383488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.384261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.384755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/groups/67da537b29d4404b8b809e3a62301b3d/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.387459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] RBAC: Authorizing `identity:create_grant(project_id=7287fa834d7d4a6181007c2bca31ad7a, group_id=67da537b29d4404b8b809e3a62301b3d, role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:44.389129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:44.416137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 198/411] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Aug 1 22:29:44 2022] PUT /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a/groups/67da537b29d4404b8b809e3a62301b3d/roles/0faca02de5824306b70c798c90d5f033 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:44.439370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f821c62-f2dd-4b21-b27c-841b827e2b54 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.484834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.485630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.485914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.486185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.486998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:44.487293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:44.488759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:44.498869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 214/412] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:44.505018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.615634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4e2b-462e-4acd-8177-c947accaf47d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.650223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.650987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.651178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.651454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] PATH_INFO: `/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.652623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] RBAC: Authorizing `identity:get_group(group_id=cc2cfb08f59748f09a88ed3098c58ce0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:44.654130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb23632-5d17-438c-8a24-cae4e115a1e5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:44.655822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 199/413] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:44 2022] GET /identity/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:44.661642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.692062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1e62-af81-43b7-9594-841a02ab2a09 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.692348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1e62-af81-43b7-9594-841a02ab2a09 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.692781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1e62-af81-43b7-9594-841a02ab2a09 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.694816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 200/414] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:44.700220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018161b-2287-43e4-8877-0e752128c956 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.700494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018161b-2287-43e4-8877-0e752128c956 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.700768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018161b-2287-43e4-8877-0e752128c956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.725026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f018161b-2287-43e4-8877-0e752128c956 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:44.726582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f018161b-2287-43e4-8877-0e752128c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:44.771939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:44.792992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:44.831019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 201/415] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:44.840616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d943b-8f80-499f-8bec-5611cda028e5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:44.876283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018161b-2287-43e4-8877-0e752128c956 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:44.890081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:44.891860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:44.892511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:44.894265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] PATH_INFO: `/v3/projects/e92bd71b1dab4dce929247e99aba6067` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:44.896529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] RBAC: Authorizing `identity:get_project(project_id=e92bd71b1dab4dce929247e99aba6067)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:44.898324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:44.902828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 215/416] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:44 2022] GET /identity/v3/projects/e92bd71b1dab4dce929247e99aba6067 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:44.910316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.025878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018161b-2287-43e4-8877-0e752128c956 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.090728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.091386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.091665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.091940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.106135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0492ae5-fd84-4033-b297-637f3e750eea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.108184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2d8dc7ca-fb6f-4540-85c0-81e96a2e6a4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:45.113408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:45.114515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:45.115303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 202/417] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:45.115984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a535-86a5-47b0-a1ba-1599d4d2fe3d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.116199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a535-86a5-47b0-a1ba-1599d4d2fe3d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.116371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a535-86a5-47b0-a1ba-1599d4d2fe3d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.117874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 203/418] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:45.123370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.123658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.123931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.145412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:45.146588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:45.155918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.156959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.157454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.158010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] PATH_INFO: `/v3/projects/e92bd71b1dab4dce929247e99aba6067/groups/cc2cfb08f59748f09a88ed3098c58ce0/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.160165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] RBAC: Authorizing `identity:create_grant(project_id=e92bd71b1dab4dce929247e99aba6067, group_id=cc2cfb08f59748f09a88ed3098c58ce0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:45.162072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:45.174431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:45.182041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 216/419] 10.222.0.47 () {58 vars in 1442 bytes} [Mon Aug 1 22:29:44 2022] PUT /identity/v3/projects/e92bd71b1dab4dce929247e99aba6067/groups/cc2cfb08f59748f09a88ed3098c58ce0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:29:45.183139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.201055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:45.270845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 204/420] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:45.351741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.364976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5bc48-e5be-4214-921a-584d44731491 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.409096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.410291 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.411571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.411571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.411571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:45.411927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:45.413216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:45.422545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 217/421] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:45.428252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.483893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8eb1-72a9-4f4c-aa3b-56909dfe13a1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.497712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:45.527307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.528589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.529316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.529845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.547661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:45.556634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:45.557509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:45.558554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:45.563680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ca61-1b70-4f89-9d8f-a6e7d2994f15 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.566479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 205/422] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:45.571121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.638007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:45.682443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.682979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.683209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.683483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.692668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b750415-2902-418a-b65a-393fb1b78678 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.731447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.732499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.733044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.733501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.734488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:45.735023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:45.736705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:45.745812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 206/423] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:45.751894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:45.834997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:45.836814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:45.889241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:45.901118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff8460-4c8c-4376-bea9-834d00342fae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:45.952735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:45.953366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:45.953667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:45.953939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:45.957892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.041292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.042791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.049339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 218/424] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:46.065065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.143176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.151560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 207/425] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:46.157989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.264545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49ed9-5443-4876-bc62-12f685f64ce4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.332148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.333056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.333508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.333944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.336450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.338083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.341639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 219/426] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:46.343880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d3968-1ef8-418d-b2cc-6152b67deff5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.362331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.409670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.410791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.411404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.411864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.411981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] PATH_INFO: `/v3/projects/2df66faea5004c4c93c8762a9ae43a4b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.414204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] RBAC: Authorizing `identity:get_project(project_id=2df66faea5004c4c93c8762a9ae43a4b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.416676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.422037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 208/427] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/projects/2df66faea5004c4c93c8762a9ae43a4b => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:46.429197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.528467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.532502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb0af-07f7-4ce7-940e-be4f49af56d4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.568393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.569293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.569732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.569962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee497f3-5ed0-494a-97a3-3993d6fcda68 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.570325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.571667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.573263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225d5d-e9a4-4937-bb45-c714bc76aea9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.580312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 220/428] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:46.581883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac30a6-f2ad-4f3c-b663-490a024447b4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.582299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac30a6-f2ad-4f3c-b663-490a024447b4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.582705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac30a6-f2ad-4f3c-b663-490a024447b4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.584356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 221/429] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:46.585173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac30a6-f2ad-4f3c-b663-490a024447b4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.607152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.607854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.608142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.608422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.609602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.611148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c3a88-ca4d-4af8-99ec-99f4f12b5303 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.613106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 209/430] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:46.614379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.614736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.615042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.633511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:46.634721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:46.691490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.700865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac30a6-f2ad-4f3c-b663-490a024447b4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.709589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 210/431] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:46.709589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.748532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.749165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.749440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.749990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.750989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:46.751380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.752954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1eaf2-e5c9-44b2-af5b-1ca54562bf9f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 222/432] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e50f-daf3-4514-ab10-fac015ce0b0d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e50f-daf3-4514-ab10-fac015ce0b0d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e50f-daf3-4514-ab10-fac015ce0b0d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 223/433] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:46.765579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489e50f-daf3-4514-ab10-fac015ce0b0d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.784262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:46.830550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf85e7-1502-4173-b802-5851fa2b6013 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.869294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.870094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.870470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.870815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.871711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=2df66faea5004c4c93c8762a9ae43a4b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:46.872088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:46.873715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3f58d-15ee-4cfb-b9dc-d801e429d842 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:46.880639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 211/434] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=2df66faea5004c4c93c8762a9ae43a4b&role.id=0faca02de5824306b70c798c90d5f033 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:46.882132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.882490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.882834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.885034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489e50f-daf3-4514-ab10-fac015ce0b0d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:46.902632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:46.903988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:46.926197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:46.926882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:46.927155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:46.927426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:46.939888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:46.945833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:46.946854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:46.948080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 224/435] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:46.954900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:46.986081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.000990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 212/436] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:47.029887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.095944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61479dee-55d6-4d0a-b1ad-a4ead6c48b72 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.117447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.140773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.141537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.141996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.142307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.143121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:47.143495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.145499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.162743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 225/437] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:47.168152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.179184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93f296-8fb6-4610-9d16-c1fd7b1ea31c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.218479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.219128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.219406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.220073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.237774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:47.242689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:47.243564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:47.244353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:47.248968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 213/438] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:47.249903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.270441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.292379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cb07b-b39c-4b19-b8ec-3a95a9fd814b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.345663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.346721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.348348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.348348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.402579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4445c27-dd5e-482a-97ca-883b652f9bc5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.420714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.447850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.448452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.448893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.449341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.449789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.450162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0762f84-13a8-434e-8a47-6eb0d89c86af None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.450823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:47.451263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.453441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.466713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 214/439] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:47.473392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.546205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.572148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 226/440] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:47.573817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f1808-e61a-4f3f-a520-09c84ae57afd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.574302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f1808-e61a-4f3f-a520-09c84ae57afd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.574761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f1808-e61a-4f3f-a520-09c84ae57afd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.576515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 227/441] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:47.578143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f1808-e61a-4f3f-a520-09c84ae57afd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.595199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488d2b-a91b-4c5a-bfd8-8ae9594beb64 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.632919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.633463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.633783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.634056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.676512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.683911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f1808-e61a-4f3f-a520-09c84ae57afd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.699869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.701404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.716753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.718020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.718020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.718020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.719146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.721002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6381d9-b722-483b-8e1a-07b5ede5c0c9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.722724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 228/442] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:47.727796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.727796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.727796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.742322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:47.743518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:47.791174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.811819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 215/443] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:47.817992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 229/444] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:47.818102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.818185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.889471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:47.912544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128c31f-b6d9-478a-8c68-3fc35691e3d3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.924616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f2141-9fcf-4eaa-be49-c857db3dd074 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:47.946742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.947376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.947656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.947923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.949158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.950660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.952239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 216/445] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:47.952982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:47.963159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:47.963811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:47.964088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:47.964360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] PATH_INFO: `/v3/projects/7287fa834d7d4a6181007c2bca31ad7a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:47.965540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=7287fa834d7d4a6181007c2bca31ad7a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:47.967005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:47.968691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 230/446] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/projects/7287fa834d7d4a6181007c2bca31ad7a => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:47.969428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.021889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.053280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dca7ec-b13d-4871-8b09-2cabeb4920dd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.072631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafb467-6f0b-4fa7-b63b-1e5190c77ee7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.086722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.087719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.088018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.088285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.099116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:48.104533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:48.105574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:48.106528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 217/447] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:48.107599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.108213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.108322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.108482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.108743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.109530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=d9a82ca5f4b244f49e8a8ba0d7999547, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:48.109857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.111391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.113669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.117352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 231/448] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:47 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=d9a82ca5f4b244f49e8a8ba0d7999547 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:48.118159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.225605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b52a9d-8b96-4fe7-a615-e799ad4fcf7a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.238017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1c8ff-1adb-45b7-9dc4-6517e73cf587 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.251790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.264565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.265222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.265498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.265819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] PATH_INFO: `/v3/groups/67da537b29d4404b8b809e3a62301b3d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.266993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] RBAC: Authorizing `identity:get_group(group_id=67da537b29d4404b8b809e3a62301b3d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.268465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.270405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 218/449] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/groups/67da537b29d4404b8b809e3a62301b3d => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:48.276018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.285079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.286149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.286910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.286910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.287766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:48.288076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.289474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.299920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 232/450] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:48.305483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.363630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.402874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e959bd-6358-4460-bb51-c82e615f288d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.422653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577c9a-4629-4514-b3bf-a5580c0c6a6e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.444827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.445466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.445806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.446082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.446881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] RBAC: Adding query filter params (group.id=67da537b29d4404b8b809e3a62301b3d, role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=7287fa834d7d4a6181007c2bca31ad7a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:48.447191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.448659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.455218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 219/451] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/role_assignments?group.id=67da537b29d4404b8b809e3a62301b3d&scope.project.id=7287fa834d7d4a6181007c2bca31ad7a&role.id=0faca02de5824306b70c798c90d5f033 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:48.464305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.465105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.465379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.465705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.516856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.545603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.547133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.653281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 233/452] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 22:29:48.659320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.763955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a6da5-ce8f-4654-8041-85b7d91a6571 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.798235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802ec7-0614-4373-8511-043762d5871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.798803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802ec7-0614-4373-8511-043762d5871e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.799077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802ec7-0614-4373-8511-043762d5871e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.799334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802ec7-0614-4373-8511-043762d5871e None admin] PATH_INFO: `/v3/projects/e92bd71b1dab4dce929247e99aba6067` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.800513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11802ec7-0614-4373-8511-043762d5871e None admin] RBAC: Authorizing `identity:get_project(project_id=e92bd71b1dab4dce929247e99aba6067)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.801978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11802ec7-0614-4373-8511-043762d5871e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.803811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 220/453] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/projects/e92bd71b1dab4dce929247e99aba6067 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:48.809677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.890547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:48.921528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f121-0eda-48aa-ab18-34e2b6726d90 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:48.959326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:48.959958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:48.960235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:48.960507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] PATH_INFO: `/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:48.961751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] RBAC: Authorizing `identity:get_group(group_id=cc2cfb08f59748f09a88ed3098c58ce0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:48.963210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3868f-39d8-44ba-9935-cb117ca75139 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:48.964813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 234/454] 10.222.0.47 () {56 vars in 1180 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/groups/cc2cfb08f59748f09a88ed3098c58ce0 => generated 229 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:48.971601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802ec7-0614-4373-8511-043762d5871e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:48.976145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:49.082093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802ec7-0614-4373-8511-043762d5871e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:49.117056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:49.117693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:49.117972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:49.118248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:49.119039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] RBAC: Adding query filter params (group.id=cc2cfb08f59748f09a88ed3098c58ce0, role.id=c512669c168542d587f922137072be5f, scope.project.id=e92bd71b1dab4dce929247e99aba6067) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:49.119352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:49.120819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb0e7-a8bd-40e1-8a9e-6649fd977f9b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:49.127289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 221/455] 10.222.0.47 () {56 vars in 1376 bytes} [Mon Aug 1 22:29:48 2022] GET /identity/v3/role_assignments?group.id=cc2cfb08f59748f09a88ed3098c58ce0&scope.project.id=e92bd71b1dab4dce929247e99aba6067&role.id=c512669c168542d587f922137072be5f => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:51.447319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4915c-eb55-4a17-bac5-66fae298ec33 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:51.448148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4915c-eb55-4a17-bac5-66fae298ec33 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:51.448576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4915c-eb55-4a17-bac5-66fae298ec33 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:51.450382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 235/456] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:51.455373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a5cf-d7ef-4cac-afc0-bf519e612ec0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:51.455791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a5cf-d7ef-4cac-afc0-bf519e612ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:51.455895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a5cf-d7ef-4cac-afc0-bf519e612ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:51.482744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1a35a5cf-d7ef-4cac-afc0-bf519e612ec0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:51.483937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a35a5cf-d7ef-4cac-afc0-bf519e612ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:51.554249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:51.565779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 222/457] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:51.583582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4915c-eb55-4a17-bac5-66fae298ec33 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:51.723374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4915c-eb55-4a17-bac5-66fae298ec33 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:51.759262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c233e16-4f23-402b-960f-028a488cc067 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:51.759888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c233e16-4f23-402b-960f-028a488cc067 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:51.760351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c233e16-4f23-402b-960f-028a488cc067 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:51.763570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 223/458] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:29:51.766151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:51.766994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:51.767270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:51.767536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:51.768248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:51.768541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:51.768819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:51.786588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:51.788272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:51.791475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:51.802107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:51.804876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:51.814017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:51.815623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:51.816585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:51.816585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:51.816585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:51.816585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:51.826294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 236/459] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:51.834062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:51.949298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 224/460] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:51.982411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cde6c-5aa4-43c2-a70a-2bfa6bfe56cc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:51.992027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.031474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.031713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.031812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.032092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.032868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:52.033178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.034636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e217c-a35d-4e2b-8186-439354fe7028 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.048938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 237/461] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:52.053817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1f7c-480a-4477-b436-4b3ee42bffd8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.053817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1f7c-480a-4477-b436-4b3ee42bffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.053817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1f7c-480a-4477-b436-4b3ee42bffd8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.053817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 238/462] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:52.057407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c1f7c-480a-4477-b436-4b3ee42bffd8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.108865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a6d08-d6eb-4f23-9813-4251f4882dc3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.119106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.137171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.145206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.146226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.146719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.147210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.158034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.159807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.169402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c1f7c-480a-4477-b436-4b3ee42bffd8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e9ae4dd1-e94b-45a0-b504-f5ebcc44c352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:52.177252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:52.178176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:52.179013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:52.179013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:52.179013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.179013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:52.181158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 225/463] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:52.182558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.183062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.183545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.202257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:52.204137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:52.205022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.205687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.205997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.206235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.206925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.209147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.241665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 239/464] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:52 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:52.241665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.305624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 226/465] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:52.334943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.447996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0410f-592f-4cd7-a339-c236854e0505 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.462259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.491098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6611e77-b8a7-4e2b-9a35-b5e3ef65a708 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.495867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.496431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.496702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.496968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.498449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:52.498449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.499379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdbf66-e38c-43fd-b070-03adfc28fa8d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.504715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.517086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 240/466] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:52.519452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae7f79-e4a9-487d-b7ed-a9b62109daa3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.519452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae7f79-e4a9-487d-b7ed-a9b62109daa3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.519452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae7f79-e4a9-487d-b7ed-a9b62109daa3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.520650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 241/467] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:52.520967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae7f79-e4a9-487d-b7ed-a9b62109daa3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.536081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.537938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.538410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.538882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.557674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.558251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-396d4477-8f24-4c32-9124-7ad4310ded07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:52.576894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:52.577929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:52.578773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:52.578773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:52.578773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.578773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:52.580667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 227/468] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:52.582005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.582420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.582911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.601784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:52.603176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:52.649070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae7f79-e4a9-487d-b7ed-a9b62109daa3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.690678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.691318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.691577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.692557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.693352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.695081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 228/469] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:52.697997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.698130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.736316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 242/470] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:52 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:52.737125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.770254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.842256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:52.855233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6389191-a413-4b7e-a436-234ed342e9e7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.856367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9613cd3-4b7c-4a79-b4d3-bbbf629fdfb3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:52.893314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.894006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.894281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.894560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.902311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:52.903696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:52.904178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:52.905030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:52.906321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:52.906548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.907130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:52.908107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.909267 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:52.924464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 229/471] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:52.924913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:52.925776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:52.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:52.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:52.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:52.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:52.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 243/472] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:52.932059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:52.933325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:53.045495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:53.061316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e24a8f-37d6-41ae-8868-0709a70e562f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.074778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60720-daa1-41b0-b609-ce84369122bf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.110375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.111097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.111380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.111704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.112396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:53.113980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6f213-14a8-4ff3-a813-2b23286929c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:53.116286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.116917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.117195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.117471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.118200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:53.118499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:53.119874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83fa9d-3fbe-4ca9-ac40-4ba470858b9d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:53.138273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 244/473] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:29:53.138273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd296-dadd-49d6-9a91-84a7f34477ca None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.138273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd296-dadd-49d6-9a91-84a7f34477ca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.138273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd296-dadd-49d6-9a91-84a7f34477ca None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.138273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 245/474] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:53.140736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dd296-dadd-49d6-9a91-84a7f34477ca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:53.144874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 230/475] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:52 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:53.146609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.147880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.148168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.171864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:53.173486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:53.176847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:53.299087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 231/476] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:53.306822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dd296-dadd-49d6-9a91-84a7f34477ca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.321766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:53.348811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.349970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.350678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.351357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.352187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:53.354538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:53.366713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:53.387353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 246/477] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:53 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:53.444388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75c06-6905-4da9-b9cc-051722c38d7f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.485424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.486361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.486786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.487200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.488151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:53.489748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:53.498811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 22:29:53.498811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 22:29:53.498811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [parameters: {'id': 'dee0122893b2484993fba0d9eb9432cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 22:29:53.498811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Aug 01 22:29:53.518624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:29:53.519851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 22:29:53.521019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 22:29:53.522145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:29:53.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:29:53.524228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': 'dee0122893b2484993fba0d9eb9432cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:53.525083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 22:29:53.525998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:53.528947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 232/478] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Aug 1 22:29:53 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:29:53.538760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:53.654428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c07a1-9e78-4713-86af-5259c58e01b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.688878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.689533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.689886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.690157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.702506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6e6233f6-22eb-4dc6-8bf0-e517e71a0aa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:53.708271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:53.709442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:53.710600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 247/479] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:29:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:53.718955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:53.723970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:53.846636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51678e-72ee-44f4-aa57-f4815fd4e98c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:53.884643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:53.885549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:53.886003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:53.886451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:53.887320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:53.887761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:53.889299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae769a3-7905-421b-b107-f7f79e37d2be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:53.898110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 233/480] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:54.698611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02ad45-25f4-41ed-9890-74bd196876da None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:54.699153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02ad45-25f4-41ed-9890-74bd196876da None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:54.699153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02ad45-25f4-41ed-9890-74bd196876da None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:54.700765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 248/481] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:54.707578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:54.708047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:54.708138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:54.734651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:54.736109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:54.797183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:54.812014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 234/482] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:54.857226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02ad45-25f4-41ed-9890-74bd196876da None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:54.927797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:54.979605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02ad45-25f4-41ed-9890-74bd196876da None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.030825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.031482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.031758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.032030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.051041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:55.056145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:55.057001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:55.057837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 249/483] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:29:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:55.064953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.178783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6016ca-1f3a-4342-b6d5-7ed1da8499c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.220482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.221123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.221404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.221700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.222434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:55.222733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:55.224127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9184f-6e33-4995-b59c-31cc42554dfb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:55.232649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 235/484] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:55.238438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.371243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16120b99-2cdb-4dc6-9050-572a6ecaf8f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.385376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:55.408075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.408424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261030b1-4c57-4250-bc62-b5d5dc219cb6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.408712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261030b1-4c57-4250-bc62-b5d5dc219cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.408987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261030b1-4c57-4250-bc62-b5d5dc219cb6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.409761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.410274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.410739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.411334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 236/485] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:55.417589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.417589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.417589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.446458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:55.447136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:55.447967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:55.448865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:55.470101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:55.471036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:55.471883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:55.471883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:55.471883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:55.471883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:55.471883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 250/486] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:55.477309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.510868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:55.552540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 237/487] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:55.595454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.601460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a810f-2ae6-4099-8fd9-13d71e50fb6d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.636984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.638063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.638533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.639031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.639960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:55.640455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:55.642242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:55.655590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 251/488] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:55.663318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.707866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:55.722367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb73543-06ce-428c-9079-b69eda300cdf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.757920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.758713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.758999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.759273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.772382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a83b2fb3-48b6-4f91-8777-ad281c4a13f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:55.778626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:55.779530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:55.780333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 238/489] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:55.782649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964ed2-eb38-4caa-a209-3fc0c9adb426 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.782649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964ed2-eb38-4caa-a209-3fc0c9adb426 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.782649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f964ed2-eb38-4caa-a209-3fc0c9adb426 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.783951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 239/490] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:55.790426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f964ed2-eb38-4caa-a209-3fc0c9adb426 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:55.827683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0279e5-7544-4885-aaf6-bdfa66b6e17c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.845164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:55.878058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.878708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.879501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.879781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.888906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:55.891010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:55.906895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f964ed2-eb38-4caa-a209-3fc0c9adb426 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-18c0a780-8217-45cd-a659-5d6bffbe4865 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:29:55.907656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:55.908752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:55.909765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 252/491] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:55.910327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.913579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.913579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.933418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:55.934693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:55.949837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:55.950467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:55.950783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:55.951160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:55.952027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:55.952331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:55.953859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:55.964238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 240/492] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:55.968636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.018418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 253/493] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:56.019062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.040753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.090539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40178c-9bc5-4534-8cb2-7457d4e0a6f3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.099980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.120731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8316c-10a1-44a8-b90b-b6cf5e99df88 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.126519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.127139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.127417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.127684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.128870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:56.129228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.131080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be80fd-291e-4c04-b09e-317f7e865bc5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.154673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.155315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.155597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.155875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.166162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.172350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 241/494] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:29:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:56.172489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.172566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53699-99ab-441b-b161-275a912dfb02 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.172566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53699-99ab-441b-b161-275a912dfb02 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.172566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53699-99ab-441b-b161-275a912dfb02 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.172566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 242/495] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:56.172566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f53699-99ab-441b-b161-275a912dfb02 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:56.189429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:56.190454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:56.191298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:56.191298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:56.191298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:56.191298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:56.191298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 254/496] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:56.192015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.263970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.285918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f53699-99ab-441b-b161-275a912dfb02 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.299516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f97e06-bd3d-4990-acf7-a3e62d8fa00d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.322233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.322880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.323155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.323421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.334617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:56.335565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.336195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.336478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.336743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cb568d99-f48a-4d9f-a051-0e5cb193da55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:56.339827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:56.340716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:56.341539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:56.345742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 243/497] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:56.347616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.347904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.348180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.365475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:56.366861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:56.387820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.411789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.413359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.445579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 244/498] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:56.445579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.484088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.524581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 255/499] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:56.525720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.553895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b244cdd-b994-498d-9445-6f44cb6b8c76 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.590606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.591230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.591544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.591780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.592474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:56.592771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.594192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.607659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 245/500] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:56.608292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.626952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.634079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdbaf9-8359-447b-ab1e-8490e6a3cf3f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.669869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.670494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.670768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.671038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.671765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:56.672057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.673462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.686017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 256/501] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:56.686017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.717315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.720601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da110f-c046-47b3-a1d6-34299520b2b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.759394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.760370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.761524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.761898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.775530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:29:56.783265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:56.784175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:56.784972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 246/502] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:56.786524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.794652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096303b-e422-4510-bc10-c3fada766640 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.828839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.829467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.829779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.830050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.840672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.842169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:56.856456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:56.857496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 257/503] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:56.863465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.863851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.885980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52152-78d2-422d-8375-ab50225f8e84 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.918925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.919564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.919841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.920115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.927434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:56.928881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:56.938121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-03ca5740-11ba-401b-a392-dcad7555e6d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:29:56.941723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:56.942835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 247/504] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc8e6-d582-474a-9dd5-9ded213cdde0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.948534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc8e6-d582-474a-9dd5-9ded213cdde0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.949213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bc8e6-d582-474a-9dd5-9ded213cdde0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:56.949213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 248/505] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:29:56.949213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bc8e6-d582-474a-9dd5-9ded213cdde0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:56.961112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d4a5-d56d-4eac-a72b-39696b41dbc9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:56.994283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:56.995013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:56.995366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:56.995714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.007045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.008539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:57.025496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:57.026570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:57.027451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:57.027451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:57.027451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.027451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:57.027451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 258/506] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:57.027794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.052899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bc8e6-d582-474a-9dd5-9ded213cdde0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.086978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.087515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.087794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.088066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.088800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:57.089148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.090522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.092734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.100298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 249/507] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:29:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:57.101204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.132751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926ef2-2ad4-4b0e-aa25-d665599e2a81 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.165374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.166052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.166322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.166586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.167295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:57.167600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.168984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2595f-c393-4778-a91f-b23cc26010e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.170445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.182264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 259/508] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:57.183468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.183751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.184027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.201926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:29:57.203170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:29:57.203382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dccab5-57f4-4faa-83a3-1e2f5b3400d7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.239676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.240293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.240574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.240844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.241678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:57.241986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.243421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.260009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 250/509] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:57.260799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.269970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 260/510] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:29:57.270857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.320559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.368145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14118-64f3-4da6-9106-ffabbb3d7583 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.377570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839c0dd-8d34-4caa-9a18-2f5c1ac7f448 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.400478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.401277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.401972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.402310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.402559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.403289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:57.403558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.405010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.411790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.412433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.412713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.412989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.416173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 251/511] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:57.417112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.423385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.424827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:57.439381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:57.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 261/512] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:57.441263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.518937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0216de-5a2a-44f5-9d15-6517bdafe1c8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.534624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.536904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dbd2-c7c8-4b6e-93a9-544689c35a3b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.551960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.552608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.552886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.553153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.560010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.561481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.569300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.569939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.570223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.570492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:29:57.573264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:57.574379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:57.575266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 252/513] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:57.575746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.621097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.644944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.646533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.673860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ead67-74aa-4246-8b5e-a977d2695668 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.707357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.707992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.708265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.708532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.719124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.720981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:57.735718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:57.736724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 253/514] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:57.737793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.748755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 262/515] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:57.749679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.770132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.825318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:57.841827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a21b-2843-474b-8bc1-7fcf4d92117f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.849810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41abb-96ab-48e7-8e99-6972f56e2753 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:57.879946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.880871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.880871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.881132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.886668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:57.887303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:57.887581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:57.887850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:57.888244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.888546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:57.888846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:57.889734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.890244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:29:57.901145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:57.902135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:29:57.903043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:57.903551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 263/516] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:57.903618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 254/517] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:57.903682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.903745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:57.988543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.000851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b4e2-1c88-4162-af28-e8d9c2698e1a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.003840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecac26-6a31-4a75-b8ab-4cec115cbb71 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.036510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.036933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.037123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.037397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.037576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.037727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.037879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.038149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.038471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.038777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.038868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.039133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.040199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.040492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.044044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.052193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 255/518] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.053001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.058722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 264/519] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.059622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.154190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad193460-ca67-4068-955f-93f1cf1ddc81 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.157928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ec6a-f417-427b-bf0d-ead51ef0a319 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.187420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.188215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.188469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.188818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.192566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.193308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.193733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.194104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.198599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.200047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.201342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.202845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.208222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:29:58.214800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:58.215852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:29:58.216737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:58.217351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:58.218190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:58.219281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:58.219281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:58.219281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:58.219281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:58.219497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 265/520] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:58.219497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.219628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 256/521] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:58.219628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.276985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.316264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea5dda-084a-4c81-a97b-023cae59798a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.321436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977810a-3666-4359-b952-d58b1d775278 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.352724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.353335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.353639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.353910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.354583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.356011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.357528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.358180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.358457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.358727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.359516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.359817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.361293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.377407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 257/522] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:29:58 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:29:58.378289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.406870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 266/523] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.406870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.452647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.505338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8733026-968c-4dd6-8bc7-65df5186c792 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.527915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c979c-516b-475b-b7ea-156ce54ffa77 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.543967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.546124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.546926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.547364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.548230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.548638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.549041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.550663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.557096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 258/524] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.558079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.569345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.570174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.570644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.570928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.571651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.571952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.573461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.591115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 267/525] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.592371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.697237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88efc5-ffe5-490b-8011-243fccab171f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.716439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038c3b9-64c1-4530-8823-a8a5f74eb133 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.720766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.737242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.738065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.738344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.738710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.739522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:58.739822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.741354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.753256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.753969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.754249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.754522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.760607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 259/526] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:58.761233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.804103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:58.831039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.832690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.871159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064b356-d2f1-4362-9619-3713c98feb26 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:58.907924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:58.908680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:58.908962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:58.909230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:58.916768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:58.918504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:29:58.930683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:58.932754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 260/527] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:58.935037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.939718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 268/528] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:58.942874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:58.989958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.027556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.060236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e9b6-3cba-4417-aaed-9c2f0f679de2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.074003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe350-067b-427b-92c3-d8d61ecdd084 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.106260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.106979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.107351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.107626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.108385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:59.108687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.110389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.111723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.112590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.113060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.113519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.129872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 269/529] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:59.129872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.188978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.190673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.230657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.241579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59cbe8-1b50-490b-880a-a1a9f9ef3a12 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.278868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.279813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.280284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.280851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.282994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.293483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.294356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 261/530] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:29:59.295981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.299824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:59.311930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:59.312804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:59.313873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:59.313873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:59.313873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:59.313873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:59.315832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 270/531] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:59.316863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.404835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837c485-39eb-4321-9ebf-a2187eb71a36 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.439609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.440070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.440413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.440690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.441533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=0f05345f84a448f3842f4e91d9f0ac67) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:59.441908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.443379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.450589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305fbe0-6974-4e7a-ab68-f65271ec2704 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.454000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 262/532] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/role_assignments?user.id=0f05345f84a448f3842f4e91d9f0ac67&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:59.455039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.475071 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.493312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.494501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.494975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.495446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.508956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.511113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:29:59.528831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:59.529790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:59.530637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:29:59.530637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:29:59.530637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:29:59.530637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:29:59.533584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 271/533] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:59.542928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.623093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.650659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f1f4-1e37-49bf-b47e-b48cb6b41981 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.691844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.691844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.692146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.692362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.692951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:59.693257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.695797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.709085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 263/534] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:59.714222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.725209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddd9fa-b3a6-423b-a11f-13bf5b90701a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.767122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.768688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.769185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.769740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.770635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:29:59.771120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.772868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.777402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:29:59.792314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 272/535] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:29:59.798954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.821449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e520-5885-4d0b-af69-852c026c5f21 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:29:59.875820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:29:59.876781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:29:59.877102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:29:59.877598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:29:59.888747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:29:59.890269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:29:59.913437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:29:59.914329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:29:59.915183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:29:59.916034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 264/536] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:29:59.924070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:29:59.996306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:00.002418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d4a98-6b00-4769-add2-8ceefe7113db None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.067996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471a6d8-e35e-4880-81d8-66ea9848de86 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.069312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.070260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.070733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.071188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.088128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:00.089983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:00.098044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:00.103779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:00.106308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:00.107146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:00.108394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 273/537] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:00.113604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:00.120267 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.121132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.121545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.122005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.122972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:00.123419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:00.125025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:00.144285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 265/538] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:00.150512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:00.253700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888f49-b1ef-4fa1-b095-d7af4bd38921 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.288755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.289686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.290107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.290522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.291461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=84a57435ddd44fe79517001c2d91eec6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:00.291910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:00.293638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:00.299747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 266/539] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/role_assignments?user.id=84a57435ddd44fe79517001c2d91eec6&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:00.311762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7f46-f3a5-42ee-9f35-4fcfc51c4adc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.359825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.359825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.359825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.359825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.360573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:00.360881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:00.362427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7a494-8696-46c0-8e76-5d12026c9a3c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:00.384748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 274/540] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:00.394780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:00.473965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:00.549735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5fad-4983-4f8e-8bae-af2af5178b36 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.601256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.601914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.602266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.602540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.603349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=4c4de72ad26844d3a0dd6d8b26159bcf) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:00.603655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:00.605099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:00.605228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d69b2-8114-44fa-8f8a-3cff48264bb2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.605424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d69b2-8114-44fa-8f8a-3cff48264bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.605737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d69b2-8114-44fa-8f8a-3cff48264bb2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.607493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 275/541] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:00.613321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3188-9440-40c2-9976-31db2c0b3aaf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.613536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3188-9440-40c2-9976-31db2c0b3aaf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.613851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3188-9440-40c2-9976-31db2c0b3aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.614225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 267/542] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/role_assignments?user.id=4c4de72ad26844d3a0dd6d8b26159bcf&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:00.634970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e02a3188-9440-40c2-9976-31db2c0b3aaf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:00.636291 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e02a3188-9440-40c2-9976-31db2c0b3aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:00.653199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:00.731288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 276/543] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:30:00.766598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:00.788396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87431-e96e-43e8-82d9-16f57582ec76 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.788921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87431-e96e-43e8-82d9-16f57582ec76 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.788921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87431-e96e-43e8-82d9-16f57582ec76 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.790616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 277/544] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:00.796132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.796406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.796686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.825984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:00.827274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:00.892077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23a376-9488-4308-bdea-9fc7bb4dcfbc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:00.914002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:00.914712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 278/545] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:30:00.933151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:00.949631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:00.950294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:00.950578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:00.950850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:00.978389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:00.988059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:00.988958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:00.989792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 268/546] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:00.997976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.058736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.107577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae6e2-940a-419d-90a6-64aab9ba639e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.144509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff938e1-95a6-48db-bfe3-a0710d2af0e4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.174866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.175898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.176422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.176876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.181578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.181578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.181578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.181578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.181902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:01.181902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.182032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.190740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 269/547] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:01.200268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:01.200625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:01.206097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:01.206974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:01.207762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:01.218007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 279/548] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:01.224059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.233808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.306867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e7ea-9ed5-4c00-bad7-bd7f287346cc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.365218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.366258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.366753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.367211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.391044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.393093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.395732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4434a-e06a-4948-8d96-2f674881bc33 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:01.419259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:01.420158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:01.421657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:01.421657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:01.421657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:01.421657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:01.424161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 270/549] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:01.429916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.430694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.431380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.431656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.431979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.432716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:01.433019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.434568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.440231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.443997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 280/550] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:01.449439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.575647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1351419-866a-4d9d-9c4d-8a4b2925311c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.578658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.612841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.613468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.613776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.614051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.614756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:01.615058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.616609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.628729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 271/551] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:01.634225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.697661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2675b52-481b-4192-8757-f08bb2fcb877 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.736619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d920f-f4b4-429e-b836-cb22dea36dc9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.756262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.757371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.757960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.758431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.776245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.776914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.777198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.777472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.778083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.779501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.785994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.787681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-100c138a-b6f1-446a-8d79-c3d0ff071322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:01.796563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:01.798015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:01.799022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:01.799022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:01.799022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:01.799022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:01.799022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 281/552] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:01.799492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8daea-b6f0-4d41-b91d-22e1cdd35811 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.799760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8daea-b6f0-4d41-b91d-22e1cdd35811 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.800027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8daea-b6f0-4d41-b91d-22e1cdd35811 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a6c66b42-9557-4cde-9ca1-e3e9a9b71f12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:01.800602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:01.802204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:01.803770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:01.804672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 282/553] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:01.804672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8daea-b6f0-4d41-b91d-22e1cdd35811 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.809186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 272/554] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:01.810743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.811027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.811299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.835308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:01.836767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:01.868284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.869926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:01.925397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 273/555] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:01.927156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:01.927777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8daea-b6f0-4d41-b91d-22e1cdd35811 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:01.965068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:01.965925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:01.966206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:01.966477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:01.967580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:01.967687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:01.969013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:01.983026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 283/556] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:01.984066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.028327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9aae-3b0b-4478-9a9b-18dd760c3714 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.062770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.063759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.064227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.064732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.065749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:02.066244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.067850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.069746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.091135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 274/557] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:02.092344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.093474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719c22-f99a-4031-a0ff-681c91283cec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.108835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.135293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.135997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.136276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.136552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.150620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:02.155670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:02.157095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 284/558] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:02.158029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.201207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22c7f-c880-46c3-a726-8339053eca9e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.236048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.237073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.237541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.238037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.245591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.247283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:02.259312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:02.260273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:02.261292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:02.263816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 275/559] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:02.264784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.268267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a7456-7fc3-409e-b827-c06ea6a42310 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.305927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.306557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.306837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.307111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.318930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.354469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.366669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548e84-6eb8-45c5-a75d-0ce2d34d98ef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.387438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.389357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.400537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.401222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.401581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.401946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.402724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:02.403102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.404512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.413049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 276/560] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:02.413975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.517724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 285/561] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:02.518874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.551264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6853fb4-4652-47bd-9473-08f56e25a883 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.565662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.589968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.590692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.591036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.591367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.592229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:02.592595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.594161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.606362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.614351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 277/562] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:02.615410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.642276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee94a9-d67e-48c8-96d2-c0f3a0967bec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.680347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.681353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.681963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.682498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.694657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.696573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:02.714380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:02.715346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:02.716226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:02.716226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:02.716226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.716226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:02.716448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce275a-48d8-42f2-973c-b8104d33e957 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.718780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 286/563] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:02.720295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.752961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.753698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.753918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.754193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.766257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.768251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:02.783702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:02.784582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:02.785507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:02.785507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:02.785507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.785507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:02.785507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 278/564] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:02.786397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.814219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.833289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb54-4df4-49c0-9bdf-934d6cc58d15 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.858953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:02.872330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.872989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.873269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.873544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.885169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.887210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.896642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851ad7-9696-4eb0-a7cb-c5abbd3ba990 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b0d454ed-540d-4266-b324-3f711971381c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:02.903779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:02.904843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:02.905795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:02.905795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:02.905795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:02.905795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:02.905795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 287/565] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:02.906502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace416f-b012-46ce-bde2-a852033d3b3f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.906783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace416f-b012-46ce-bde2-a852033d3b3f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.907055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace416f-b012-46ce-bde2-a852033d3b3f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.913134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 288/566] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:02.913946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace416f-b012-46ce-bde2-a852033d3b3f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:02.940359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:02.942057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:02.942489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:02.942905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:02.943746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:02.944192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:02.945785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:02.958113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 279/567] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:02.958973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.034797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace416f-b012-46ce-bde2-a852033d3b3f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.060859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.068609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.069441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.069755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.070028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.070719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.071023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.072397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dcf68-2541-4c0c-807d-cebb0a5c05f2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.077397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2cc13-0233-4f6d-aa7c-fd1ba1f1022c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.083881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 289/568] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.085041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a1fa-1ee9-4ee9-ac66-8afac863a07e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.085322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a1fa-1ee9-4ee9-ac66-8afac863a07e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.085617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322a1fa-1ee9-4ee9-ac66-8afac863a07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.103005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0322a1fa-1ee9-4ee9-ac66-8afac863a07e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:03.104456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0322a1fa-1ee9-4ee9-ac66-8afac863a07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:03.111666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.112285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.112584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.112827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.113521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.113851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.115208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.126815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 280/569] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.127604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.136120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.173871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 290/570] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:03.175123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab62c2-abbf-4a20-b934-028b1627e436 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.175430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab62c2-abbf-4a20-b934-028b1627e436 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.175673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab62c2-abbf-4a20-b934-028b1627e436 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.176930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 291/571] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:03.181250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab62c2-abbf-4a20-b934-028b1627e436 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.233104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa63176-384e-4ba2-a924-d232896959c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.266320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.266993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.267283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.267560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.275656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.277112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:03.290994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:03.292389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:03.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 281/572] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:03.294693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.298725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.302062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab62c2-abbf-4a20-b934-028b1627e436 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.337597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.338279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.338556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.338826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.346150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.347626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4f1f8db3-1c7f-4b8b-b282-52d060fb88df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:03.358646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:03.359512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:03.360412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 292/573] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:03.361302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf039-3392-4638-ac86-d92639f69556 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.361594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf039-3392-4638-ac86-d92639f69556 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.361888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf039-3392-4638-ac86-d92639f69556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.379018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-113bf039-3392-4638-ac86-d92639f69556 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:03.380209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-113bf039-3392-4638-ac86-d92639f69556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:03.399758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618bd6-0675-4cd3-a2bd-b11b535408cd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.399908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.434666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.435347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.435599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.435874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.442920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.444370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.445760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 293/574] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:03.450432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf039-3392-4638-ac86-d92639f69556 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:03.455915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:03.456830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 282/575] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:03.457796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.519763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.556146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf039-3392-4638-ac86-d92639f69556 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.558210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79388-ca0c-4b6f-a9e6-b5b619199a43 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.590897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.591542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.591814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.592083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.592568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.593189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.593487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.593778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.594583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.594916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.596700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.602330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:03.607413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:03.608332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:03.609174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 294/576] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:03.609899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.617107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 283/577] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.617956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.633209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.715116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d430b2e-f00d-4b82-ad2e-414ca0818c86 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.715347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53e08a-c927-46ce-96b3-0384ff9839d9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.737872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.751255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.751884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.752162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.752437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.752538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.753112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.753214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.753387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.753497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.753686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.754425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.754723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.755003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.756126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.766526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 284/578] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.767493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.768739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 295/579] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.769605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.844731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:03.866905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3599d5f-44b0-4571-b70a-5f5c9731e91c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.867485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ac13-a7a9-4089-910e-164253dd107f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:03.900733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.900907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:03.901381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.901482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:03.901707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.901800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:03.901985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.902090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:03.902692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:03.902987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:03.904364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:03.913498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:03.916409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 296/580] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:03.917210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:03.919080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:03.920225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:03.921215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 285/581] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:03.922248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:03.965265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.022087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73d75-9570-4eee-ba51-49aade7dbfff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.029975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b11b9-99ea-4a18-bfff-2809f493167b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.055179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.055851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.056132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.056391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.063459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.065188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.065982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.066168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.066470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.068673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.070348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.080423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=b74c1e8e8d3643db9230f448af705d3a, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.081885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:04.084519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:04.085531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:04.086634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:04.086634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:04.086634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.086634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:04.086634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 297/582] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:04.087050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.109151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 286/583] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:03 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:04.109151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.204170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.211009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d923-418a-4d88-bdf8-733079d92aad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.214689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2cea0-6aa2-4193-ab1a-e66f87cf6d29 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.250597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.251299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.251573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.251840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.252644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=223d4641075e4b6489e3ee1e4fbdce36) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:04.252949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.254111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.254440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.254555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.254726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.255002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.263799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 298/584] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/role_assignments?user.id=223d4641075e4b6489e3ee1e4fbdce36&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:04.264646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.270973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.272448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:04.291099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:04.292255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:04.293261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:04.293261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:04.293261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.293261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:04.293261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 287/585] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:04.293678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.340985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.399202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ceb2-ecae-4c0f-a270-120058c82af0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.441645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.442088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a82d1f-5edc-4a98-8974-2a50ea9f3eeb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.442274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.442558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.442824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.443556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:04.443859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.446271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.460851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 299/586] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:04.462017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.487405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.488189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.488566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.488938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.489902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:04.490595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.492089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.507844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 288/587] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:04.512427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.517035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.579949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b5f3-2eb6-415b-b6ad-405d527897ca None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.599899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.619411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.620186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.620565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.620938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.621777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:04.622302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.623874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.632329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e9f9-d7a7-4e48-ae5c-f5db25afc84e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.637294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 300/588] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:04.638282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.669661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.670619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.670905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.671175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.683050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.684599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:04.700567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:04.701858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:04.703245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:04.703245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:04.703245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:04.703245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:04.706630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 289/589] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:04.707545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.754844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094e9d7-07d8-48f4-bd57-09dbd683bcb5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.770032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.793187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.794309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.794791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.795290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.803217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.805043 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.816196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2ea71-c8eb-4bac-9ffb-7508aa3c9b65 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:04.819653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:04.820551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:04.821447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:04.824141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 301/590] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:04.825337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.858927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.859584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.859866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.860199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.866425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:04.869536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.871117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:04.884252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:04.885285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:04.886272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 290/591] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:04.886952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:04.951664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b251d77-3069-40d3-8dcb-d1ddfbb4e7f8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:04.990311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:04.991325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:04.991817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:04.992280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:04.993238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:04.993758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:04.995443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:04.999027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda3cc-ce8d-4cbd-ba2a-6c0a09b30b0b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.011886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 302/592] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:05.013132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.025957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.041886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.042659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.043034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.043398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.044382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:05.044874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.046431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.065103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 291/593] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:05.066194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.115642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.121675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a97c4-1eb5-4c00-8489-e6079c25f302 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.155711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.156683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.157200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.158115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.159160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:05.159722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.161352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.170439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a6a1c-06c8-4266-9bfe-0c267b8c8f6a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.184080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 303/594] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:05.185213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.206083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.206668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.207130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.207407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.215162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.216662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:05.229620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:05.230696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:30:05.231665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:05.235558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 292/595] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:05.236566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.292247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351bf6-159f-489b-a707-23268eb3f7aa None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.298096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.331305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.332527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.333009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.333478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/0f05345f84a448f3842f4e91d9f0ac67/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.353359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=0f05345f84a448f3842f4e91d9f0ac67, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.355039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.362986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f79e-232d-4509-a086-61607e0bedeb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.377599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 304/596] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:05 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/0f05345f84a448f3842f4e91d9f0ac67/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:05.378718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.383115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.435848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.436991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.438144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.438858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.463594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.467767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:05.500096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:05.501356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:05.502476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:05.502476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:05.502476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:05.502476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:05.504149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 293/597] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:05.511821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.574086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a2bf3-a757-43f2-8b28-f1bdf3a20b02 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.623754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.624658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.625593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.626073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.627299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:05.627746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.630321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.652711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 305/598] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:05.654453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.660450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.665830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12056bbb-512e-4eb8-9941-8f00e1a32c2c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.750775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:05.762286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.762920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.763200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.763582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.764295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:05.764591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.766003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.793953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 294/599] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:05.802234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.828533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae508f5-366c-4a6d-9fd8-ea4b44f1e1b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:05.893754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:05.894447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:05.894671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:05.894937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:05.916456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:05.918363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:05.944446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:05.945416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:05.946351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:05.946351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:05.946351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:05.946351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:05.946351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 306/600] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:05.956830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:05.959969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebfcdc-bac9-4d2f-8999-2e267b97b968 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.001980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.003556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.004832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.005326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.023162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.024645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:06.054566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:06.055520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:06.056427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:06.061034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 295/601] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:06.067110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.088949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.162209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b043-3b9b-4664-9394-21ec1bbdad13 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.182839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.189403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9af4ea-7e9a-444d-91d8-c3706826d824 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.226790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.228190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.229033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.229776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.231195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:06.231958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.232445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.233216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.234847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.235281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.235751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.236623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:06.238895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.240407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc391e-1506-4ea1-b830-58de1961e59b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.260765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 296/602] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:06.268352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 307/603] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:06.274098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.278527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299fd0-c3eb-4e8b-a6b7-58d44c6162b9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.278981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299fd0-c3eb-4e8b-a6b7-58d44c6162b9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.279395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299fd0-c3eb-4e8b-a6b7-58d44c6162b9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.282156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 308/604] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:06.283138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299fd0-c3eb-4e8b-a6b7-58d44c6162b9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.354999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.394572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172c899-b6eb-45c9-ae4a-be5a3c79c6d5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.400084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299fd0-c3eb-4e8b-a6b7-58d44c6162b9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.432371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.433294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.433757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.434258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.437131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.437963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.438247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.438597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/84a57435ddd44fe79517001c2d91eec6/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.442940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.444560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.454102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=84a57435ddd44fe79517001c2d91eec6, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.455558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2efc67b8-9e42-4b48-84d8-79d75b07ce08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:06.457533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:06.458617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:06.459599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:06.461693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 297/605] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:06.463104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080499c8-858a-436c-a09e-f7f734afde46 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.463513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080499c8-858a-436c-a09e-f7f734afde46 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.463917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080499c8-858a-436c-a09e-f7f734afde46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.481108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 309/606] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:06 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/84a57435ddd44fe79517001c2d91eec6/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:06.482135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.485125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-080499c8-858a-436c-a09e-f7f734afde46 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:06.486813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-080499c8-858a-436c-a09e-f7f734afde46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:06.510405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.568446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 298/607] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:06.569887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d90d9-ded9-4f7f-8ffe-0fee986ce888 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.570179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d90d9-ded9-4f7f-8ffe-0fee986ce888 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.570453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d90d9-ded9-4f7f-8ffe-0fee986ce888 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.575966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 299/608] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:06.578253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.578535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.578809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.598154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:06.599377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:06.623499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd726a2-006a-4d59-a8ec-395b1f938282 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.664100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.685697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.686343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.686620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.686893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.687429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 300/609] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:06.688125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.688314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:06.688678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.690264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.710075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 310/610] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:06.712777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.808872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:06.810843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a794-9a0c-46ee-ba8a-3a354e89c478 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.826276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179f2a-839c-41ee-a8d3-74d3469880c9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:06.850653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.851590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.852015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.852569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.861496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:06.862072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:06.862375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:06.862748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/4c4de72ad26844d3a0dd6d8b26159bcf/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:06.866016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:06.871101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:06.872229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:06.873239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:06.875719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 301/611] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:06.876661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.882887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=4c4de72ad26844d3a0dd6d8b26159bcf, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:06.885004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:06.905122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 311/612] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:06 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/4c4de72ad26844d3a0dd6d8b26159bcf/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:06.910557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:06.991533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.041502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e80bf8-0d9a-4b5c-94cc-daf6bce9f17e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.083805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.084717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.085145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.085595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.086554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:07.086999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.089204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.091054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.102776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 312/613] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:07.115798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.148477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb226482-4f69-4fcf-a9a4-f0ea9fe6d7b5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.195312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.195967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.196249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.196521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.211121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:07.216391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:07.217573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:07.218521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:07.220528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 302/614] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:07.224808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.243679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7b6fe-366f-4beb-88ad-88b5fe24b408 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.287662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.289148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.289594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.290279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.305533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.308739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:07.328316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:07.329355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:07.330294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:07.330294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:07.330294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:07.330294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:07.333642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 313/615] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:07.342643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.347829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb86db-5528-4247-b8b1-c1f171e61327 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.358001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.386260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.387307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.387799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.388303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.389467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:07.390005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.391688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.405696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 303/616] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:07.408559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.435630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.519812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012ebbc-fb2b-4e90-8a13-8d7e59e24a14 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.524798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4b39-de61-4e8f-a4cb-555f3577b1a1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.555871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.556993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.557605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.558112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.587918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.590779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.593348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.594511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.596737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.597255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.598228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:07.599931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.601512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906ceba-d50b-4ed2-9f84-31c36b86cf9e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:07.624050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:07.625060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:07.626125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:07.626125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:07.626125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:07.626125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:07.637392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 314/617] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:07.641490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a19b9-d4f4-48c0-b69b-e8b0817c8880 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.641490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a19b9-d4f4-48c0-b69b-e8b0817c8880 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.641490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a19b9-d4f4-48c0-b69b-e8b0817c8880 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.641490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 315/619] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:07.642156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 304/619] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:07.645763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a19b9-d4f4-48c0-b69b-e8b0817c8880 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.646416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.674036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.772150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a19b9-d4f4-48c0-b69b-e8b0817c8880 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.780722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238434-d55e-4783-b338-63b3691bc6b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:07.782467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.817823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.819144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.820011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.820749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.827829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:07.828451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.828729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.828999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.829770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:07.830076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.831453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497bee8-204b-4fd4-8d8f-1fe72dd511f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.831977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:07.834619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:07.848750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 305/620] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:07.850030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:07.850321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:07.850593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:07.851119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:07.852198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:07.853145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:07.859852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 316/621] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:07.859852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.873950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:07.875322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:07.949976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 306/622] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:07.951003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:07.972443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:07.997388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94151d1-30c4-4fcf-935c-8d02efe72daf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.053187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.054156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.054881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.055302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.066116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.068265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.074781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe363-f5f8-42b3-a399-3ec9833e1afe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:08.082455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:08.083549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:08.084671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:08.086729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 317/623] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:08.087687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.117099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.127201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.128063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.128481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.128889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.129850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:08.130288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.132385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.156561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 307/624] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:08.158167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.210451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49575d30-db93-45d3-975a-1513721b7bc5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.239662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.257025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.258221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.259009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.259440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.273862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:08.274721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784c179-5b9f-4097-be53-c101fba9d0b9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:08.280523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:08.281527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:08.282686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:08.292169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 318/625] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:08.292169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.321992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.322703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.322963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.323240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.324286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:08.324404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.326053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.349657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 308/626] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:08.351314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.392090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.408249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4305c-352a-4f2b-9d49-33bf2ded131e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.445103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.445773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.446056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.446379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.484480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c86e37-59e7-47bc-b0fe-84d38ee4529c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.516666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.518178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.518500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.529726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.530975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.531806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.532521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.533533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:08.534026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.536118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.551804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 309/627] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:08.551804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.634100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.651167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 319/628] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:08.652319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.662272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9505b-19c0-472c-85a7-1177245611a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.698267 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.699102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.699440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.699645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.711355 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.712853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a6c20ec3-d269-4cb6-b387-484efb693576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:08.728231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:08.729123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:08.730048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:08.730048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:08.730048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.730048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:08.730048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 310/629] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:08.731668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e80-d9c4-43d2-b4a2-f7ba718f44c8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.731668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e80-d9c4-43d2-b4a2-f7ba718f44c8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.731668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e80-d9c4-43d2-b4a2-f7ba718f44c8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.732383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 311/630] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:08.733128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4e80-d9c4-43d2-b4a2-f7ba718f44c8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.773353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd9fb3-ebb9-423d-9fb4-28f0e9caca05 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.778740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.808129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.808820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.809100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.809377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.819651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.821122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:08.835965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:08.836898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 320/631] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:08.838148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:08.854395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4e80-d9c4-43d2-b4a2-f7ba718f44c8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.884817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:08.890990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.891688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.891975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.892302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.903057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.904478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-818b21d9-af00-440a-b71f-f0ef1d6b7bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:08.920056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:08.920992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:08.922568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:08.922568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:08.922568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:08.922568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:08.922568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 312/632] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:08.922919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.923138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.923625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.942819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:08.944045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:08.949306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567fa9c-5ff9-4da8-9bca-4b68677e76f0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:08.984102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:08.984730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:08.985011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:08.985285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:08.986096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:08.986402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:08.987786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:08.999537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 321/633] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.001758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.030382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 313/634] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:09.030382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.049048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.120832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.124127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dce89a-bfe8-48bb-8059-27a4d493dc72 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.127135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca8bdf-6e74-479e-a578-f6240cbb154e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.162198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.164773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.164773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.164773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.164773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:09.164773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.167241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.187695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.189353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.190340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.190853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.191765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:09.195221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.203952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 322/635] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.203952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.205068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.232108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 314/636] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.233335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.322797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc788-b420-48f8-bcab-e5fe68e234b4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.337170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.354088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528fb4b-0efb-4ef4-bdca-f7512e37e069 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.360757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.361958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.362311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.362729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.370512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.372108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:09.384083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:09.385202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:09.386307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:09.388253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 323/637] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:09.389272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.392128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.392788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.393155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.394190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.404584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:09.410867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:09.411993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:09.412990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 315/638] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:09.413842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.418031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.506535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1060-dce2-45e9-b0c9-f40334918529 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.542286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.542933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.543205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.543468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.551375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.552835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6873046e-4e56-4431-8edd-8993bbf37783 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:09.563818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:09.565008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 22:30:09.566211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:09.569522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 324/639] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:09.570785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd7fe-ac5a-4637-b7b0-1a9321ee8264 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.571069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd7fe-ac5a-4637-b7b0-1a9321ee8264 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.571340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd7fe-ac5a-4637-b7b0-1a9321ee8264 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.572600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 325/640] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:09.577221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd7fe-ac5a-4637-b7b0-1a9321ee8264 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.641757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305132d-7e92-4a9a-b733-32faad35147a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.659791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.676025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.676586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.676865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.677135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.683660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd7fe-ac5a-4637-b7b0-1a9321ee8264 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.683799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.685255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:09.696620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:09.697600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:09.698609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 316/641] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:09.699086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.702217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.724063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.724690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.724966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.725245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.726033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:09.726336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.727785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.739882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 326/642] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.742744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.800303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baee507-c887-47f1-b325-ad11c27d79bb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.833255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.833999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.834283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.834558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.835285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:09.835583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.836905 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3736fb5-29f5-43b5-9335-dfca1b16e14d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:09.837007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8206be6-72b0-4893-a50b-47d70c9a3a3f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.845735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 317/643] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.846885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.847166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.847667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.864605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:09.865833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:09.870059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:09.870670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:09.870942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:09.871207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:09.872000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:09.872301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:09.873763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:09.882422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.891202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 327/644] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:09.892045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:09.925677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:09.933422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 318/645] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:09.934200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.007609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cecdc-1417-4317-ad27-cc37c2103d9b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.039588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67d9b6-6471-4055-993a-99cb4edf82ad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.042410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.043027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.043310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.043584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.044365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:10.044669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.046147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.057695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 328/646] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:10.058486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.073674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.074369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.074641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.074921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/223d4641075e4b6489e3ee1e4fbdce36/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.089374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=223d4641075e4b6489e3ee1e4fbdce36, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.090867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.103881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 319/647] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:09 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/223d4641075e4b6489e3ee1e4fbdce36/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:10.117690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.121751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.160066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.176368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e5ef-6ca4-4b4d-89ab-f6ab3aec6c8f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.216182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.216853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.217135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.217410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.218517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4e4f3-2f58-4312-844b-a9ff017a2134 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.231143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.232681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:10.250266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:10.251396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:10.252297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:10.252297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:10.252297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:10.252297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:10.252297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 329/648] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:10.252581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.267898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.268539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.268807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.269077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.342700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.344266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.365782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376afce-af9d-41bb-8f1c-08deb279598f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.376929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.390527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.403143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.403793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.404067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.404344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.415319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:10.420698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:10.421669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:10.422500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 330/649] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:10.423319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.454004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 320/650] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:10.454994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.535133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151e0a-bb1f-4f18-9e12-2046e574c705 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.555153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7ab2c-ada0-4223-b283-cc6233297e53 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.576234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.576857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.577185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.577454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.578215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:10.578557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.580012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.586957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 331/651] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:10.590061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.590718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.590978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.591242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.591958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:10.592262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.593715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.594874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.612345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 321/652] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:10.613425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.624194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.648249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.731556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fe124-d164-4034-9d82-95a3af73dfc6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.746186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662065-2ceb-4079-a498-1fde033a580b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.770973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.771624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.771903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.772235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.782830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.783756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.784895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.785021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.785545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.785890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.786707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:10.787098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:10.788543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:10.804955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:10.805951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:10.806875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:10.806875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:10.806875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:10.806875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:10.810366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 332/653] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:10.811432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.813232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 322/654] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:10.816351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:10.897963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.929035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:10.936965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d176aa-eb69-42ff-85d4-66c548ed602d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.950568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2563-97c7-43ca-8b7f-447e82b96b3b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:10.995233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.996234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.996656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.996770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:10.997205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:10.997318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:10.997596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:10.997873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.007105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.008558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.012375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.014042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:11.021064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:11.022077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:11.023017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:11.027061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 333/655] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:11.028215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:11.034203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:11.035110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:11.036040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:11.036040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:11.036040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:11.036040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:11.040075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 323/656] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:11.041242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.148529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111163-17c0-4534-9da3-9fd7ad2ff432 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.167761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4020-9292-4631-9a49-fb9e3db8150b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.179817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.204267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.204909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.205188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.205455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.206259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:11.206561 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.207951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.217116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.218127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.218127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.218324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.219092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:11.219641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.221211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.225527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 334/657] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:11.226697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.240184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 324/658] 10.222.0.47 () {56 vars in 1137 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:11.241104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.248682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.348041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d4152-865b-48aa-833d-657724ba5d33 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.361176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd1a4-6af0-44a9-81ce-9a2ffaf7b534 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.382983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.383639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.383918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.384188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.391828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.393447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.397336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.397988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.398263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.398534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.399226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:11.399522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.400879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:11.406062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:11.406949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:11.407898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 325/659] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:11.412701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.417034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 335/660] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:11.417993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.445230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.502630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.522438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353fc9d-592b-4c1d-bb80-49f6565589c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.526618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23ac1-d2a1-44b0-9e45-4a8394c7855b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.559103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.559703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.559980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.560308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.562747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.563674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.564134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.564538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.565423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:11.565903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.567470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.580859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 336/661] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:11.581881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.631069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.632441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.661306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.687312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f981f-4731-4d54-99f6-656b2b891808 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.724254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.725157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.725639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.726048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.729609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.736249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.737879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.751041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 326/662] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:11.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:11.752684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 22:30:11.753879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:11.755517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.760231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 337/663] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:11.760584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.866686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bcaac-fa2b-41a6-a46f-c16da65e3c72 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.887032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa718d-df0f-453c-9769-f0bae12bcba1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:11.899225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:11.909604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.910207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.910485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.910754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.911551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=0f05345f84a448f3842f4e91d9f0ac67) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:11.911850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.913778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:11.925643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 327/664] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/role_assignments?user.id=0f05345f84a448f3842f4e91d9f0ac67&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:11.926740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:11.942375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:11.943322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:11.943761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:11.944175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:11.970971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:11.976120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:12.015020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:12.016436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:12.017929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:12.017929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:12.017929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:12.017929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:12.017929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 338/665] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:12.030682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.071318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cecbc4-9c52-47e8-878c-8539b1ab0dfb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.109293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.110243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.110749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.111164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.112115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:12.112562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.114214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.138757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 328/666] 10.222.0.47 () {56 vars in 1139 bytes} [Mon Aug 1 22:30:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:12.144077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.159097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.204362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.225609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08415c6e-2040-4a3e-aaf5-a0e8ee446ca2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.264912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fda15e-9ecc-47da-88e2-bb255b03ecdd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.271732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.272622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.272889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.273209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.274865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:12.275157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.278910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd081-2151-4a04-84b4-90150bc3b964 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.292452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 339/667] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:12.298344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf3b-92f2-4b4c-ba6e-52a82f40d5d1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.298877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf3b-92f2-4b4c-ba6e-52a82f40d5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.299635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf3b-92f2-4b4c-ba6e-52a82f40d5d1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.301900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 340/668] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:12.303145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadf3b-92f2-4b4c-ba6e-52a82f40d5d1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.314570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.315528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.316051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.316503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.403291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.404662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.422427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadf3b-92f2-4b4c-ba6e-52a82f40d5d1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.432213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.458190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.458851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.459132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.459407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.462122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.466566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.468028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-efde48bc-07e1-4e80-938e-afed02c696e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:12.480391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:12.481409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:12.482395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 341/669] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:12.483370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.483370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.483679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.501812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:12.503362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:12.523223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 329/670] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:12.524324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.581768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 342/671] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:12.582629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.656401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543aa7-4445-4821-8981-f876a81a94ec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.699677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.700601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.700882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.701410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.702618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.712219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d436da-7854-4a47-a0d4-a417bb95e2b0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.713330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:12.715785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.717801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.734322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 330/672] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:12.735239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.750413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:12.751654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.752345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.752736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.753144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.768488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.770002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:12.790146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:12.791304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:12.792506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:12.792506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:12.792506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:12.792506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:12.792506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 343/673] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:12.792795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.858493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab508-2a03-4414-8080-9dc08b710ab7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.893089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.893760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.894090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.894344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.899380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbe8c9-652c-4172-829a-7bcec8a8d06e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:12.904742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:12.910407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:12.911555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 331/674] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:12.912415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.940519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:12.941150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.941432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.941767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.942560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=84a57435ddd44fe79517001c2d91eec6) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:12.942872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:12.944323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5611a-9640-44e7-977c-565f7a1d2b49 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:12.950394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 344/675] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/role_assignments?user.id=84a57435ddd44fe79517001c2d91eec6&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:12.955678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3196-577e-42e3-a47a-fba0a57a3a99 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:12.956211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3196-577e-42e3-a47a-fba0a57a3a99 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:12.956557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3196-577e-42e3-a47a-fba0a57a3a99 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:12.961723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 345/676] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:12.963120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb3196-577e-42e3-a47a-fba0a57a3a99 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:12.983823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.004459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.052427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4544-64a8-4ae7-9042-ff67edf33f5e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.088734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb3196-577e-42e3-a47a-fba0a57a3a99 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.102927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.103667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.104006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.104344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.105282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.105674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.107801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb42f2-1cb6-4242-bb96-9789924da3e6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.125706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 332/677] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.127447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.128019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.129273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.139143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.140880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.141189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.141466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.142361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.142671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.144038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.152693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:13.154236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:13.160131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 346/678] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.160131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.237416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 333/679] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:13.238561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.248700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.286009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b24d-6b84-4234-8bfd-9452997dd4ef None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.326016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.326562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.326989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.327873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.328171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.338357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.340208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:13.354252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:13.355180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:13.356187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 347/680] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:13.356685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.374715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a5ca-03e8-4a64-a419-3afa5fa8fa3c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.418070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.419328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.419618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.420161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.433831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.435301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:13.456011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:13.458260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 334/681] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:13.460443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.466579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67940487-61d4-439e-8233-5b3012c1925c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.502918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.503485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.503753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.504012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.507212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.517749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:13.523032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:13.525159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 348/682] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:13.527229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.576538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149c8e-9457-42ac-bb5a-fc60d41c756a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.589803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.613208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.614142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.614631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.615049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.616000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.616440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.618045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.630850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 335/683] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.632016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.648793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b05b9-7600-4e21-bfe9-f860b97b94f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.693071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.693734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.694015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.694296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.695254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.695958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.697660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.720543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 349/684] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.720543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.743116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f841c8-1607-4dc3-b221-92fcb8a7e6ba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.779528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.780529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.781012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.781485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.782643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.783451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.785701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.788104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.797340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 336/685] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.798665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.836125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:13.846108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6e6a5-19a3-4a4a-b076-9cfa3d06e858 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:13.882986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:13.884011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:13.884479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:13.884933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:13.886088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=4c4de72ad26844d3a0dd6d8b26159bcf) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:13.886665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:13.888413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:13.895218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 350/686] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/role_assignments?user.id=4c4de72ad26844d3a0dd6d8b26159bcf&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:13.896380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:13.925727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ade24-cfc3-47a5-9244-197dd8649ca2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.009107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.012658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.013154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.013657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.029205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.035304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:14.059182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:14.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:14.066296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 337/687] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:14.069839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.081015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b34c3-e53c-4724-9973-1b4e748c3ccf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.115775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.148835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.149505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.149825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.150094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.166605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.168118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:14.190712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:14.191683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:14.192595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:14.192595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:14.192595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:14.192595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:14.192595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 351/688] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:14.197894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.229979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.238813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a7341-3a8e-4ff1-b8d0-15a8f2499d5e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.279120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.280178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.280748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.281249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.282417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:14.282933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.284564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aedc93-e0e0-45df-9bbc-5e7501b2c605 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.304403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 338/689] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:14.306213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263cd5-4848-4066-a49b-bedfdbb0caa4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.306722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263cd5-4848-4066-a49b-bedfdbb0caa4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.307189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a263cd5-4848-4066-a49b-bedfdbb0caa4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.308949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 339/690] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:14.311944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a263cd5-4848-4066-a49b-bedfdbb0caa4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.327635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c6f8c-6a15-4c8b-9746-bbc6f7ee3d17 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.363596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.364221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.364506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.364780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.365477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:14.365803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.367180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d547-fc0a-433e-b4e2-09a72fd333f6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.380160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 352/691] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:14.381383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.386040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.386349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.405512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:14.406799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:14.430902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a263cd5-4848-4066-a49b-bedfdbb0caa4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.434085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.479729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.480361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.480634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.480902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.482981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 353/692] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:14.484049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.522165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.579731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.582024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.621874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b72430-2e56-457b-a0d4-99e816c29b69 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.671368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.671725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.671998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.672255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.683632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.683632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:14.699637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:14.700619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:14.701547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 354/693] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:14.706511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.731770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 340/694] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:14.733002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.742840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.809700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:14.837666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc450a57-a20a-4567-b881-0815d060ac57 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.866413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7883-a3b5-4e9c-8d60-9c605db5d6a9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:14.880901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.881908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.882378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.882853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.894693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.896483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.902303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:14.902926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:14.903203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:14.903472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:14.904274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:14.904572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:14.906177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:14.920808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:14.922281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:14.923285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:14.923285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:14.923285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:14.923285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:14.930163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 355/695] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:14.931342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:14.948276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 341/696] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:14.948276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.026820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.050436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3756a-452a-4d12-bb24-9b758e89ee6f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.055992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e414-6916-4f59-b45d-24be82b44bc9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.061510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.085794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.086415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.086692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.086961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.087732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:15.087938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.089306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.098109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.098965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.099402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.099674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.101483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 342/697] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:15.102557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.111288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.112856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-11e43163-d22c-45bf-ab6d-b14d55bf6f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:15.128479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:15.129928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 356/698] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:15.130870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7235-db5d-4099-9007-6456ded3da63 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.131216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7235-db5d-4099-9007-6456ded3da63 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.131739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7235-db5d-4099-9007-6456ded3da63 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.136617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 357/699] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:15.136617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7235-db5d-4099-9007-6456ded3da63 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.242574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab409-5cd2-477b-8da5-29a625027d8a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.280156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7235-db5d-4099-9007-6456ded3da63 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.281813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.282913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.283252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.283597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.309446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.319109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.319742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.320023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.320300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.321238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.322481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.332068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.351044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 358/700] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:30:15 2022] POST /identity/v3/users => generated 254 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:15.351044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.371146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.372672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6525f-2206-4183-b323-c1d0a16d3862 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.490450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367acb7-60d9-4075-b76b-4dab0ef964dc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.499733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 343/701] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:15.502220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.502509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.502792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.536382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:15.537598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:15.538104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.539020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.539404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.539769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.540737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:15.541136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.542788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.564138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 359/702] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:15.564138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.611300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.616608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.629580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 344/703] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:15.629580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.693434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d15a7e-4038-4ec5-9ce6-5b2a51899b40 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.736342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.737009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.737287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.737627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.748591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.749823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73653c-6604-47f4-80f8-aa1db8873f05 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.750146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:15.773062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:15.774227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:15.775375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:15.775375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:15.775375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:15.775375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:15.775654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 360/704] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:15.776505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.788335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.789129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.789511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.789912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.797067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.798785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:15.810782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:15.811925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:15.813109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:15.814846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 345/705] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:15.815780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.876777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.894326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:15.903468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff51b0-1123-41bb-9f14-2053abb8191e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.919304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5bd95-0a71-4165-b1e9-9a84721a9362 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:15.940964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.941933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.942351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.942785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.943655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.945252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.955317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:15.956095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:15.956471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:15.956841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:15.957671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:15.958074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:15.958809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:15.959970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:15.960997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:15.962491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:15.963825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 361/706] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:15.964968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:15.981771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 346/707] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:15.982834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.086938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb57e3f-7df8-4b07-9649-af40e1fc56bd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.098448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a6a4-c278-462f-b8f9-34fd7683900b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.118804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:16.129695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.131138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.131701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.132151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.133179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.133691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.133812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.134476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.134820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.135161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.135932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.136203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.136304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.137767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.142883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 347/708] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.143771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.150618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 362/709] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.151782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.173675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:16.246789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43706466-37a5-42d4-8c25-5f219957a7df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.268769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc9021-fd3b-4017-97ee-2e60a46d93e7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.281980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.282855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.283136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.283409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.290170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.291628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.302369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:16.302982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:16.303858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:16.304749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:16.305178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.305178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.305178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.306005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.306156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.308272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51c5b1-d07a-452c-8db1-7fa3164ff2cc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.309167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 348/710] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:16.310086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 363/711] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7563c-a0f3-4b6e-922e-7f9ec6edd73e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7563c-a0f3-4b6e-922e-7f9ec6edd73e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7563c-a0f3-4b6e-922e-7f9ec6edd73e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 364/712] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:16.329630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7563c-a0f3-4b6e-922e-7f9ec6edd73e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.366594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:16.440779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:16.448870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbb95-5324-440d-93d4-b1fc4523255d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.466220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7563c-a0f3-4b6e-922e-7f9ec6edd73e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.497037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.497735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.498023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.498295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.499101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=223d4641075e4b6489e3ee1e4fbdce36) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.499405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.500855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61423d6-9c95-4bb7-b556-64cdc98963a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.509482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 349/713] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/role_assignments?user.id=223d4641075e4b6489e3ee1e4fbdce36&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.510856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.511144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.511418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.524366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.525001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.525279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.525548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.526588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.526896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.528500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.536904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:16.538476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:16.686603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 365/714] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.696212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.740753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 350/715] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:16.768814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.809083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f34c7-751b-4c4c-9e1f-7d796dae2d3f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.810375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:16.847035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.847938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.848358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.848816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.849786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:16.850250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.851830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.858116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 366/716] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:16.890699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cfc6b-1649-41b7-a56b-dd73d4fda525 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:16.934511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:16.935531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:16.935960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:16.936392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:16.937183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:16.938858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:16.961767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:16.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:16.964077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:16.966718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 351/717] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:16.973230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:16.983284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:17.106349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80585a12-5232-479a-978d-8c3f2eddea65 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:17.142068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:17.142748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:17.143024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:17.145578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:17.145578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] RBAC: Adding query filter params (name=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:17.145578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:17.145578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82891146-8163-4c8f-ad81-95fab6598f63 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:17.159876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 367/718] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:17.171200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:17.371255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa81e-d627-4a78-901d-9261a09d648b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:17.373898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:17.428224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:17.428880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:17.429159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:17.429431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:17.430190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:17.430495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:17.433305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45088f16-0406-48a2-8ece-d61602e571f9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:17.447075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 352/719] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:17.684723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dd5a6-ab92-47cf-83db-0eba20677b4b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:17.685409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dd5a6-ab92-47cf-83db-0eba20677b4b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:17.685919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dd5a6-ab92-47cf-83db-0eba20677b4b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:17.688596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 368/720] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:17.693004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:17.693502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:17.693625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:17.719628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:17.722471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:17.810897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 353/721] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:17.835817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dd5a6-ab92-47cf-83db-0eba20677b4b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:17.981917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:18.103225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dd5a6-ab92-47cf-83db-0eba20677b4b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:18.160136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:18.160784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.162605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.162605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.183891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:18.189340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:18.190311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:18.191187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:18.199391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 369/722] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:18.200614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:18.381317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:18.389175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd499b8-00cf-439a-9436-81c58968a04c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:18.448225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:18.448884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.449166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.449444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.450235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:18.450531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:18.451927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e5fe-d7e2-4d95-960f-ec1071236593 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:18.464634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 354/723] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:18.470939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:18.609106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07867c8-5a6a-4f6e-9833-d38206b1a87c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.609489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07867c8-5a6a-4f6e-9833-d38206b1a87c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.609724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07867c8-5a6a-4f6e-9833-d38206b1a87c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.611424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 355/724] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:18.616786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.617085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.617339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.647081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:18.648209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:18.694823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb391a-e79a-49be-a066-01415b534714 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:18.762194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:18.771311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:18.772152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.772594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.772877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.783597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 356/725] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 22:30:18.796259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:18.797734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:18.812935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:18.813943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:18.814856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:18.814856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:18.814856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:18.814856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:18.818570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 370/726] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:18.823924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:18.845270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:18.957681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95163e-d426-4e77-94f5-22695926584d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:18.990926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:18.994689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:18.995359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:18.999772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:18.999772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:18.999772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:18.999772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:19.011218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be915c1-5838-464f-9db7-a1330d8ccaf4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.037537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 371/727] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:18 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:19.097112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.097793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.098078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.098356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.099074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:19.099375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:19.100783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84f23b-aa64-4b12-a6e0-33e444d5b2d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:19.128222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 357/728] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:19.133837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.182500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:19.246193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087e3a-b244-4b8c-8eb7-81c96a34686f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.256003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76f6f-10f5-4d79-ab3c-04f1d6c67e3b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.256574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76f6f-10f5-4d79-ab3c-04f1d6c67e3b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.256992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76f6f-10f5-4d79-ab3c-04f1d6c67e3b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.258571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 358/729] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:19.264955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04d08-4bce-4537-9805-59927c57804c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.265375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04d08-4bce-4537-9805-59927c57804c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.265891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04d08-4bce-4537-9805-59927c57804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.290714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.291702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.292142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.292573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.301274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:19.303765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:19.310729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a3d04d08-4bce-4537-9805-59927c57804c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:19.312033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3d04d08-4bce-4537-9805-59927c57804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:19.316907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:19.317926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:19.318928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:19.321756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 372/730] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:19.332640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.416634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 359/731] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:19.417999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c741d-8815-41c2-a4ab-4395b4bd282a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.418290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c741d-8815-41c2-a4ab-4395b4bd282a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.419287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c741d-8815-41c2-a4ab-4395b4bd282a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.420769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 360/732] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:19.443527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.444065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.444344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.460729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ffffc-94c6-4514-a8c1-c28cbfee0375 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.462473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:19.464912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:19.466951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:19.507606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.509595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.509595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.509595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.509595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:19.509887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:19.511358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:19.531407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 373/733] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:19.532369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.546283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 361/734] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:19.547166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.551801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:19.658271 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dacb8e-d97d-4d2f-ab3c-105d443bbc33 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.663968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e7c94-9919-40da-8d18-53a9cb3570ca None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.697587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.697587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.697587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.697587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.702058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.704391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.704391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.704391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.715530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:19.721300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:19.722271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:19.723112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:19.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 374/735] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:19.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.755833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:19.794594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:19.796803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:19.800248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:19.890243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624d57c-497f-4dd1-a79d-a87fa461f910 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:19.938314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:19.938954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:19.939336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 362/736] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 22:30:19.939452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:19.939791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:19.940792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:19.954992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:19.960022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:19.960982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:19.962233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:19.966122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 375/737] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:19.966961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.071791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214802b-7528-40d2-bd6f-9113ee3cce92 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.075547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.104712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.119598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.120319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.120598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.121030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.122087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:20.122446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.123969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7be2f-370a-4be5-8776-472e4ba9dd47 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.141741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 363/738] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:20.141741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44c6e8-096f-434e-b4c6-42fec75e17ce None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.141741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44c6e8-096f-434e-b4c6-42fec75e17ce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.141741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44c6e8-096f-434e-b4c6-42fec75e17ce None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.144774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 364/739] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:20.145378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44c6e8-096f-434e-b4c6-42fec75e17ce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.150877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46d47d-39a3-4f38-b549-d4ed36c96ac7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.191159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.191797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.192140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.192343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.207099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.208703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:20.224582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:20.225479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 376/740] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:20.231335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.299053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44c6e8-096f-434e-b4c6-42fec75e17ce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.343555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.344134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.344459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.344830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.345840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:20.346148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.347648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c169e5-b7e6-4614-97ce-6ef27fc4e5e4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.357709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 365/741] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:20.359006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.359374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.359713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.364256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15374529-e5ef-4eaa-b3c1-38723d963671 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.386356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:20.387549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:20.413581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.418619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.425269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.425969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.426247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.426521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.446109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.447573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:20.467680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:20.468609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:20.469500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:20.469500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:20.469500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.469500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:20.475135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 377/742] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:20.475831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.483781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 366/743] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 22:30:20.490078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.619514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342597f-14cf-4844-a424-8cae31c184a3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.624478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238c01-04d8-4af5-933f-b52e99f618a5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.663963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.664705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.665135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.665403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.666208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:20.666502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.667927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.668727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.669870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.670825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.671306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.683341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.685151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.685261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 378/744] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:20.686102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:20.703870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:20.705322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:20.706358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:20.706358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:20.706358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:20.706358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:20.708734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 367/745] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:20.710518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.733420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.747433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:20.813083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19853-26f5-40e5-ad03-e97c3b028236 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.837667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba573d-9985-45d8-8a76-02ae39758eca None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:20.853470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.854710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.855307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.856076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.857250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:20.857770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.859940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.872830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:20.873649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 379/746] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:20.874713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:20.878084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:20.878392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:20.878669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:20.879395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:20.880798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:20.910067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 368/747] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:20 2022] POST /identity/v3/services => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:20.911243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.094296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde761e-714e-4893-bfdc-19c6ed49d372 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.100491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.120582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.183697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe951313-b0d4-42f0-9dcd-dbad2ce47358 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.186197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.188302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.189835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.190785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.213917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.218487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:21.234078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:21.235001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:21.235951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:21.239249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 380/748] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:21.240905 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.268645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.272339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.275127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.277266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.280691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:21.281008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.286150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.348331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 369/749] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:21.351565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.469346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7129-4a2b-4ac9-967a-2eaa10b9c432 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.505594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1e07f-5fb3-4051-bbb7-1a9ef26ad5f7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.525501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.526319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.526700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.527096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.535326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.536878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-66280bca-e3d5-4312-a9cd-e213eff3715a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:21.548696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:21.550023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:21.551139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:21.551664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.552309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 381/750] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:21.553735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b811-f778-4d9b-984d-df272db14f00 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.554012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b811-f778-4d9b-984d-df272db14f00 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.554278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b811-f778-4d9b-984d-df272db14f00 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.555881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 382/751] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:21.557382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2b811-f778-4d9b-984d-df272db14f00 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.572324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.573086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.573457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.573930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.578452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:21.578770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.580354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.593351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 370/752] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:21.599169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.607972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.730231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0219d-ab84-4a53-ae05-305480722e80 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.732453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2b811-f778-4d9b-984d-df272db14f00 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.770144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.770797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.771106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.771339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.774734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.775607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.776114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.776576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.777586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:21.778102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.779375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.780243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.780809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cfbc8880-bbab-4d17-83fd-e76a86ab0a62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:21.792891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:21.793963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:21.794893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:21.798883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 383/753] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:21.799939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.800232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.800505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.807286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 371/754] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:21.807286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.823751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:21.824941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:21.849237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.898592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 384/755] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:21.899764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:21.912062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:21.919661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bbbca-5062-4a13-b2aa-b129e95b60e1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:21.954858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:21.955479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:21.955756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:21.956024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:21.956811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=feed14a81a6146338abb5ed7f7726e6c) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:21.957115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:21.958585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:21.964351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 372/756] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/role_assignments?user.id=feed14a81a6146338abb5ed7f7726e6c&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:21.965268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.060154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818710-7282-4f52-bc80-9364249c9251 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.109587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.110455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.110831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.111204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.112151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:22.112588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.114572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.122745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014c05-c9fb-46c3-a024-e18c0df02945 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.134503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.143749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 385/757] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:22.143749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.175862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.177010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.177387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.177704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.226385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.265472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.267017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.273494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9c290-3274-4585-afab-7138571ae077 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.318464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.319748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.320172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.320609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.321396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.323025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:22.334457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:22.335415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:22.336281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:22.341380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 386/758] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:22.342614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.395010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 373/759] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:30:22.396396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.409872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.483362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdd091-021e-42a9-959b-f4491db764c5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.517399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d155056-280f-47a0-84be-4bdc634c487d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.519024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.525400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.526078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.526353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.526623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.541276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.542767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:22.563690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:22.564633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:22.565501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:22.565501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:22.565501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:22.565501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:22.569783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 387/760] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:22.570739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.575432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.576331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.576839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.577301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.578204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:22.578697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.580962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.592770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 374/761] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:22.592770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.674010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.676519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d80d48-ccbb-4f40-bfdc-9b6101af1722 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.708776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd60aa-f574-480e-aa03-f3495573fd7c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.713284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.713975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.714253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.714522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.726083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.727540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:22.742403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:22.743349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:22.744238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:22.744238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:22.744238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:22.744238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:22.744238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 388/762] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:22.744947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.752567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.753491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.754088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.754567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.755522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:22.756135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.757842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.773968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 375/763] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:22.775099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.785974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.853236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f18cff-dd90-4c1d-ad01-ab991222e8fc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.884965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c2c2-a846-43c8-9348-371c194042f1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:22.892901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.892901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.893140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.893525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.894339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:22.894734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.896183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.906718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 389/764] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:22.906718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:22.926996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:22.932509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:22.933535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:22.933997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:22.934422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:22.935256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:22.935817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:22.937502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:22.950969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 376/765] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:22.952285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.018036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.021837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29775ce-9db9-4d13-951a-34cc01454d27 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.060677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.061403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.061750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6be4e-3300-47c4-94c8-a8c0e108b177 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.061867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.062270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.069762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.071447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:23.084234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:23.085414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:23.086501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 390/766] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:23.090500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.101812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.102531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.103215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.103554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.104690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] RBAC: Adding query filter params (interface=public, service_id=9edd14a32b224b4bbf75a513385490e7, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:23.105064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.106644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e0321-1d7c-4a06-b930-2fae4b6e7abd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 377/767] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:22 2022] GET /identity/v3/endpoints?service_id=9edd14a32b224b4bbf75a513385490e7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae2c-30de-4dcd-be19-787da9fe0c2e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae2c-30de-4dcd-be19-787da9fe0c2e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae2c-30de-4dcd-be19-787da9fe0c2e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 378/768] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:23.120418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ae2c-30de-4dcd-be19-787da9fe0c2e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.182359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.207989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91062-0c54-4bfc-8190-22fc27d1b503 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.232729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ae2c-30de-4dcd-be19-787da9fe0c2e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.245265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.245978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.246219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.246486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.253389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.254868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:23.266412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:23.267436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:23.268339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 391/769] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:23.268819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.276483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.277175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.277533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.277892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.278711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:23.279074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.280561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13dc6ea-3fd0-4938-a914-4df47eedd4bd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.293092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 379/770] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:23.296489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.299480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.299768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.300028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.317858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:23.318969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:23.383703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198e8d2-bca4-426c-95ce-ad1db7f922fb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.388744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 380/771] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:23.394807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.418600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.419231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.419509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.419782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.420442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.421838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.429949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 392/772] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:23.429949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.455412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.519025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d894d7c-4cad-4699-8326-5eefc6664a47 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.560229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.581078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.581397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c5b0-7795-4ce5-8523-28894c9ac1ff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.582321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.582808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.583344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.584464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:23.584978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.586685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.614045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 381/773] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:23.615209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.659337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.659337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.659337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.659707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.662664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=b74c1e8e8d3643db9230f448af705d3a, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.664447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.693898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 393/774] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:23 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:23.694984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.836340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f8fb1-0a06-4182-ac38-fed4bdeaf018 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.853189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:23.910663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.911325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.911583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.911854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.912488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.912885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2a70a-6352-4dfa-80db-486f1d73bac6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:23.914714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:23.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:23.927401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:23.928252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:23.932547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 382/775] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/services/network => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:23.934573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:23.957618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:23.958632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:23.959206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:23.959626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:23.960579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=223d4641075e4b6489e3ee1e4fbdce36) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:23.961052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:23.962997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7706-3b95-4696-82a5-4f253c7431bb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:23.972164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 394/776] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/role_assignments?user.id=223d4641075e4b6489e3ee1e4fbdce36&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.056958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2a09-5585-49e8-a957-a039ea5b4319 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.057204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2a09-5585-49e8-a957-a039ea5b4319 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.057484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2a09-5585-49e8-a957-a039ea5b4319 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.057712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2bc-736b-4027-9951-62227097ae0a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.060002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 395/777] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:24.067357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.067691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.068090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.102467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.103471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.103471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.103736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.104232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:24.104531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:24.106186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:24.114108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:24.115760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:24.116087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 383/778] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.117455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:24.118637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:24.166319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:24.237271 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb5847-2a31-4172-b74d-718a26f947be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.293516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 396/779] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:24.296014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.296823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.297232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.305786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.307171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:24.307676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:24.309458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:24.325756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 384/780] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.334921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:24.338379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:24.491079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:24.496074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31165-2899-4c99-a4c9-68292c42fb2c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.580698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd997c-a88b-4b69-ae67-51eec7bc5bab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.585762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.586422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.586697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.586967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.608019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:24.613597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:24.614552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:24.615396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:24.619106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 385/781] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:24.626691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:24.626997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.627635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.627912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.628183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.628954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] RBAC: Adding query filter params (interface=public, service_id=962ad238c8024ee1a1d58b216d273edf, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:24.629257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:24.630671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:24.638698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 397/782] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/endpoints?service_id=962ad238c8024ee1a1d58b216d273edf&interface=public®ion_id=RegionOne => generated 194 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.644282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:24.649821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:24.808379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908970e2-3dd0-4622-aada-72fc7e8ef617 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.840394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab747c-1ddf-4390-b56c-59812c84dd8f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:24.850233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.851097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.855872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.856314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.857120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:24.858691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:24.874019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 398/783] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.923749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:24.925021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:24.925440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:24.925875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:24.926882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:24.927322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:24.928845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:24.958318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 386/784] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:24.963920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:25.098059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:25.146488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85f2ba-0880-4f9a-8eb1-1e94f0a1ad39 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:25.186464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:25.187119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:25.187405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:25.187682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:25.200242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:25.201723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:25.219330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:25.220355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:25.221818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:25.221818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:25.221818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:25.221818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:25.221818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 399/785] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:25.226667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:25.252692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:25.417401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05668c-e360-401b-82fd-1b5bcc606a36 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:25.484987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:25.485761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:25.486044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:25.486315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:25.487017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:25.487318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:25.488699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:25.509607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 387/786] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:25.515244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:25.677493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef0a52-15d1-49e1-820d-31aa514666b9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:25.705119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:25.760176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:25.760901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:25.761182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:25.761454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:25.776386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:25.777863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6e47d42d-5f25-435c-bc9c-fc3428928567 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:25.795238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:25.796213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:25.797609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 400/787] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:25.807059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:25.951570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.019647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61646a48-2c74-4257-b980-8ac1293c5582 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.065271 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.066646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.067069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.067174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.069777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:26.069777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.069777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.092999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7929350-f6f7-4098-8d02-16663108e2fd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.093337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7929350-f6f7-4098-8d02-16663108e2fd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.093572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7929350-f6f7-4098-8d02-16663108e2fd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.096326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 401/788] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:26.101937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfa18-7a08-4f52-8b2d-ed11e346da6b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.102218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfa18-7a08-4f52-8b2d-ed11e346da6b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.102495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 388/789] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:26.102702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfa18-7a08-4f52-8b2d-ed11e346da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.110679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.133781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-be7dfa18-7a08-4f52-8b2d-ed11e346da6b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:26.134978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be7dfa18-7a08-4f52-8b2d-ed11e346da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:26.264273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 402/790] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:26.273872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a667-eeef-4383-8fd2-9967d8a5b98e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.274182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a667-eeef-4383-8fd2-9967d8a5b98e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.274464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a667-eeef-4383-8fd2-9967d8a5b98e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.279834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 403/791] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:26.281312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6546d4-4721-49e9-b4df-9ae7c8572f86 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.289494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71668-7e2b-48b4-a130-15f3239a927f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.289794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71668-7e2b-48b4-a130-15f3239a927f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.290072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71668-7e2b-48b4-a130-15f3239a927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.316450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0ee71668-7e2b-48b4-a130-15f3239a927f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:26.317730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ee71668-7e2b-48b4-a130-15f3239a927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:26.324718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.325845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.326344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.326806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.334660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.339736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.342214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:26.361336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:26.362395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:26.363311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:26.363311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:26.363311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:26.363311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:26.363311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 389/792] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:26.364205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.395667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 404/793] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:26.397030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15106-db59-45cf-8481-d72f7ca1f09b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.397312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15106-db59-45cf-8481-d72f7ca1f09b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.397604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15106-db59-45cf-8481-d72f7ca1f09b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.399147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 405/794] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:26.399846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15106-db59-45cf-8481-d72f7ca1f09b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.452018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.480409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2c6a5-60de-46b2-a182-8d67658ebb2d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.516335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15106-db59-45cf-8481-d72f7ca1f09b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.520065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.520959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.521429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.521930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.522747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.524721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5ff95092-e055-4098-bd5b-3174a3fc7d75 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:26.534765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:26.535697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 22:30:26.536588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:26.539253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 390/795] 10.222.0.47 () {56 vars in 1129 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:26.540492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.540772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.541037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.555220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.555915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.556257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.556592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.557348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:26.557744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.559300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.560107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:26.561330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:26.573150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 406/796] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:26.573865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.625084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.644697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 391/797] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:26.645465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.682412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.687133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41ed54-50d4-437f-8486-38361b087204 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.726002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.726650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.726928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.727200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.739182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:26.744512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:26.745438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:26.746367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 407/798] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:26.747062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.756075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5dab7-0557-442e-b812-a533334c4158 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.790120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.790661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.790936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.791202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.791881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:26.792185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.793549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.798300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 392/799] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:26.798984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.851638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.868432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0d29-cfc5-4b0e-b460-fccafca4ce1a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.904677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.905777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.906118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.906451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.909990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df564b92-d825-4578-aab9-3e48abdcd77c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:26.913263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:26.914740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:26.926409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:26.927404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:26.928289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 408/800] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:26.928759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:26.951470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:26.952121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:26.952657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:26.952657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:26.955086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:26.965252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:26.971221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:26.972178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:26.973038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 393/801] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:26.974597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.043210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79d506-9c02-4261-af27-9324c842429f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.083335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.083982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.084253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.084518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.085229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.085524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.086922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.092889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a8100-5b85-4649-9daa-831a97d4ca02 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.095273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 409/802] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:27.096410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.106830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.136562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.137615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.137919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.138186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.139292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.139607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.141289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.146398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 394/803] 10.222.0.47 () {56 vars in 1124 bytes} [Mon Aug 1 22:30:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:27.147183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.185832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.208117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cbda8-3c0d-4719-b9fc-47ed80b9abb2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.249387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.250126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.250446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.250714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.251461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.251747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.253186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.263065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe748d-6f2a-4432-a973-127fa0287d88 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.271113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 410/804] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:27.271878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.298609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.299173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.299438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.299709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.300429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.300732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.302152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.310020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 395/805] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:27.310779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.343866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.376411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b536f-9a7d-4c9e-a2f3-505215bf17a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.409914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.410558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.410828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.411096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.413130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d033d9-3890-4053-a2bb-f6a426cbae97 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.419240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.421796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.423567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2bc39c7c-4b35-40a6-bd1a-eb880f14ae14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:27.438524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:27.439513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:27.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:27.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:27.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:27.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:27.440406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 411/806] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:27.440900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391836-7558-47f4-b84b-3fca479ebb8f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.441170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391836-7558-47f4-b84b-3fca479ebb8f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.441450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9391836-7558-47f4-b84b-3fca479ebb8f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.442684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 412/807] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:27.443350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9391836-7558-47f4-b84b-3fca479ebb8f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.453070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.453744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.454030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.454301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.455022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.456418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.479574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 396/808] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:27.480422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.559308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9391836-7558-47f4-b84b-3fca479ebb8f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.597951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.599430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.600088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.600437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.600660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/feed14a81a6146338abb5ed7f7726e6c/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.613240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d9621-df35-48a5-a5e2-efc47182eb9f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.619654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=feed14a81a6146338abb5ed7f7726e6c, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.621240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.639869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 413/809] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:27 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/feed14a81a6146338abb5ed7f7726e6c/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:27.641637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.651900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.652577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.652860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.653137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.674165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.675680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-73ab0e96-4f31-48aa-9b96-0a57a1147e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:27.695194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:27.696158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:27.697168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:27.697168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:27.697168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:27.697168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:27.697168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 397/810] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:27.698793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.698793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.698793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.708463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.719989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:27.721117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:27.787810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d1b23-2e3a-4fcc-90ea-5ba1a30a4aac None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.821233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 398/811] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:27.821411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.840273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.841122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.841323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.841609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.842361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.842666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.844180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:27.857541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 414/812] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:27.858549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:27.917947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.949993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe67f8-84c2-4ac2-9550-71a1993144a7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.965792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:27.970027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8a9ed-8b4a-4785-b63b-ff21b836ed3d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:27.988128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:27.988757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:27.989044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:27.989321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:27.990065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:27.990372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:27.991775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.008063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.008974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.009420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.009951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] PATH_INFO: `/v3/services/network` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.010737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.012279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.021495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 399/813] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:28.021495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:28.027850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:28.028835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 22:30:28.029694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:28.032586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 415/814] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:27 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:28.033604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.131283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94468be-c8aa-412d-a660-837f1f68a99a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.145223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b67391-5d53-4ead-a87d-14ef5d70da34 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.162168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.169268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.170030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.170314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.170588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.178480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.179880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.182829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.183800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.184284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.184766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:28.191859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:28.192804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:28.194151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:28.196145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.197843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.197968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 400/815] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:28.198718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:28.210541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:28.211501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:28.212467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:28.215067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 416/816] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:28.216067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.248629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.310604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315f3a0-691c-44af-858b-3a664ec2fa53 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.328445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540638b4-1c47-41d2-9a51-8fe2162b30fe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.349119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.349788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.350441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.350716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.351415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:28.351720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.355030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.360500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 401/817] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:28.361433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.375833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.376913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.377397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.377959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.379057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:28.379651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.381461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.401855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 417/818] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:28.403063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.474068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5836b-c832-4497-9db1-00a8a30f7a8f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.503770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.509187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.509980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.510270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.510665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.511487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:28.511797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.513240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.527335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f364d-42b5-43e0-bee2-5eb21bda36d9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.537924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 402/819] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:28.538828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.573257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.574605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.575118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.575635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.576625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] RBAC: Adding query filter params (type=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:28.577128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.578779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.585028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 418/820] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:28.586282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.643169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad219e9e-bb84-4496-a812-bd54718b36ba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.657745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.687386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.688023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.688305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.688680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.702537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286116-80db-4942-9e1e-1adc20342cea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.743876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.744941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.745418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.745917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.758094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.759922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.763365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.763526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.764942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:28.780075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:28.781062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:28.782007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:28.782007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:28.782007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:28.782007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:28.784168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 419/821] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:28.785363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.889300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 403/822] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:30:28.890551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:28.907718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263edef-045d-4e90-b7b9-29c30b055765 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:28.909957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:28.946715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:28.947675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:28.948161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:28.948626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:28.949678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:28.951269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:28.979599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 420/823] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:28.981465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.032275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857a8c1-bf75-416e-a32e-c7cc7cae2041 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.050871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.121746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0a4cc-36b9-4b15-9c75-46a78ad862d7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.126486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.127142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.127426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.127702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.128422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:29.128732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.130171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.166970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 404/824] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:29.167491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.168397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.168874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.169357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.172976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.183756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.185743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:29.203347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:29.204461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:29.205383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:29.205383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:29.205383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:29.205383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:29.208619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 421/825] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:29.214886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.306695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.342415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d50c-9a38-41c7-a054-58c137b61d38 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.368149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.371943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea229-c3ec-4356-888c-e9fcd482893b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.386948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.387609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.387892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.388170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.397267 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.398777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:29.411139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:29.412102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:29.413009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 405/826] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:29.418681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.420536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.421512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.422041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.422599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.423695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:29.424196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.425883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.441534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 422/827] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:29.447218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.535904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73a3d-458d-415c-b633-ec0c1ea06759 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.562072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8b369-a8d4-4cc5-b80b-5815a8937153 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.571731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.577976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.579111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.579656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.579932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.580702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:29.581008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.582498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.596487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 406/828] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:29.602326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.604875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.606343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.606824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.607301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.615194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.617016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:29.630064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:29.631109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:29.632005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:29.634720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 423/829] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:29.640453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.667836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.716028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d8aea-8d2d-4591-aa07-8682aefbc343 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.751879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f49bc-824f-4d26-98be-4a2ee0887fa3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:29.756125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.757070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.757632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.758110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/223d4641075e4b6489e3ee1e4fbdce36/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.760402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=223d4641075e4b6489e3ee1e4fbdce36, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.762251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139482e-f193-4ffd-99bb-fba4a0176367 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.778528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 407/830] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:29 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/223d4641075e4b6489e3ee1e4fbdce36/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:30:29.779845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e2d-fa8f-4683-a745-4e2feb73798a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.780130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e2d-fa8f-4683-a745-4e2feb73798a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.780398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e2d-fa8f-4683-a745-4e2feb73798a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.781807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 408/831] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:29.782906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f15b-74fb-48e8-ad29-e968f8c92144 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.783191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f15b-74fb-48e8-ad29-e968f8c92144 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.783461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f15b-74fb-48e8-ad29-e968f8c92144 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.784873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 409/832] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:29.790544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.790879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.791039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:29.791151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.791661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.791937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.792208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.792982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:29.793288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:29.794906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b0c9c-63d9-4c8b-b679-399cd5313e90 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:29.814607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 424/833] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:29.814853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:29.814956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:29.814956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:29.814956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:29.815189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:29.831617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:29.832988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:29.866928 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:29.901975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 410/834] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:29.902927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.916970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 425/835] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:29.924973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:29.939261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.030705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8579-dc51-4e3a-b096-0fbafe9d0170 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.050548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3da932-f8ab-46b1-965c-49a53e00b305 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.073136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.074272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.074555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.074829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.075633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:30.075942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.077385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.090438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 411/836] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:30.090438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.090733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.090733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.090733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.090733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.091026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.091105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:30.107403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:30.108492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:30.109327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:30.114336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 426/837] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:30.115333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.140203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.211348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666a5f9-bfe4-429c-af6c-9d5f8da0af21 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.216756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.253753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.254675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.255131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.255318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bdd74-a3d7-4646-949f-383a4ddbd41b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.255740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.268982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:30.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:30.276028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:30.277065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 412/838] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:30.281906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.305475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.306177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.306461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.306732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.307420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:30.307720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.309094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.321606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 427/839] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:30.322964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.430361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb76-c5b1-4bb2-8ac7-cbb8e139b1cc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.433626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.453416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc808-f014-4b33-a9a0-fca0da6d57b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.478560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.479447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.479730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.480241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.481396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:30.481734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.483712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.489401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.490056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.490770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.491045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.492000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:30.492298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.493705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.494142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 413/840] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:30.499285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 428/841] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:30.499435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.614722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725a4f4-4f72-40c8-b85b-1a5774097587 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.657583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.657583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.657583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.657583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.670416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.671746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:30.686376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:30.687308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:30.688651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:30.688651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:30.688651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:30.688651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:30.688651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 414/842] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:30.693399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.697595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.803034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a9aa-bb10-4e19-9d3f-71ae20a0d8df None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:30.836893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:30.837527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:30.837845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:30.838119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:30.838824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:30.839122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:30.840500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:30.853442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 429/843] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:30.858781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:30.881244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:30.972910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bb3de-7717-4944-be74-97085b9474a1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:31.010263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:31.010877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.011159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.011463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.023256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:31.024950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:31.036748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:31.037790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:31.038764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:31.041356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 415/844] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:31.046874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:31.118789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:31.164225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050df2e-26e4-4181-a16e-bb0a77dc4bd1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:31.199324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:31.200100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.200503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.200861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.201778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:31.202178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:31.203736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:31.223028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 430/845] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:31.228943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:31.311269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:31.336710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9dd5-2233-4143-8b91-4d0bd33e7f3e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:31.371398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:31.372023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.372303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.372575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.439770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:31.441152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:31.525676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:31.545485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 416/846] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 22:30:31.554750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:31.666954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fddbb4-2631-4d44-9930-7e4ea4d3da7a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:31.690957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:31.707674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:31.709081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.709580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.709989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.722345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:31.723989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6c50ceec-9234-4da0-93ca-1c65cd252303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:31.739917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:31.741052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:31.742039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:31.742039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:31.742039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:31.742039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:31.743864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 431/847] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:31.750713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:31.858588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6194e-238b-4032-aea9-5e06eea42944 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:31.880683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accb29a-a98a-45bc-a8fb-1621a859d3fc None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.880960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accb29a-a98a-45bc-a8fb-1621a859d3fc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.881233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accb29a-a98a-45bc-a8fb-1621a859d3fc None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.882763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 432/848] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:31.889650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.889936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.890206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.900019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:31.900847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:31.901288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:31.901733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:31.902815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:31.904644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:31.906578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:31.910335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:31.911511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:31.921578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 417/849] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:31.929641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:31.979876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.001460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 433/850] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:32.034880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.067421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d045f8-f7fc-4c61-a9dd-7a15eb44f059 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.112504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.113473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.114232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.114681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.124110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.126467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:32.145979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:32.146966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:32.147952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:32.150698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 418/851] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:32.158037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.164425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6e26-c875-4928-9737-264e854d4d77 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.166370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.216522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.217678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.218059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.218430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.232136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:32.238708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:32.240114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:30:32.241148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:32.248513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 434/852] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:32.251188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.294069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.299303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d4fa4-16fb-40fe-a373-0918b8975173 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.335324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.336068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.336678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.336951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.337755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:32.338341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.339916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576dc9-fcb6-4bc3-a631-b6ed5ebe9676 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.357423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 419/853] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:32.359109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b426-df22-4c71-8d0e-d8da164b1de7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.359392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b426-df22-4c71-8d0e-d8da164b1de7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.359664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b426-df22-4c71-8d0e-d8da164b1de7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.361064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 420/854] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:32.363405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b426-df22-4c71-8d0e-d8da164b1de7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.370124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeccda-97e9-48b6-aaa0-ffed098861e6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.408995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.409748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.410179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.410538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.411422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:32.411811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.413281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5150ac0-5c31-494e-abf5-1a5a9cf2701d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 435/855] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:32.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.442039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:32.443229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:32.463478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.478187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b426-df22-4c71-8d0e-d8da164b1de7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.514237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.514772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.514994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 436/856] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:32.521939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.521939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.522107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.522180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=feed14a81a6146338abb5ed7f7726e6c) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:32.522180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.522180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.538781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 421/857] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/role_assignments?user.id=feed14a81a6146338abb5ed7f7726e6c&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:32.539691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.576765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.645409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d848ed-d8c1-4b91-b860-b91006cba825 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.666572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea54a2-49c7-4028-acba-ed3ffc3b8c35 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.686525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.687139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.687414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.687684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.699413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.700799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.706031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.706843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.707947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.707947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.708513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.710014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e373f475-42cd-4d6b-a9e5-39033c4e15fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:32.716916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:32.718345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:32.719383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:32.719383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:32.719383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:32.719383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:32.719383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 437/858] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:32.720548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ddb9-0431-4731-a031-3ed429e7e489 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.721176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ddb9-0431-4731-a031-3ed429e7e489 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.721439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ddb9-0431-4731-a031-3ed429e7e489 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6e7b476a-022c-4df9-9f27-9b3b39c9f619 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:32.722215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:32.726332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:32.727418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:32.728464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 438/859] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:32.728539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 422/860] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:32.729144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ddb9-0431-4731-a031-3ed429e7e489 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.730090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.730512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.730794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.739643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.749963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:32.751210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:32.828784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 423/861] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:32.830009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.850511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.862982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ddb9-0431-4731-a031-3ed429e7e489 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.901067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.901725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.902004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.902275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.903074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:32.903289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.904641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.916962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 439/862] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:32.916962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:32.943590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02de0-3030-49e9-83fa-85f84c24ff7b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:32.979019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:32.979641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:32.979915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:32.980241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:32.980986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:32.981287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:32.982712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:32.984923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:32.987685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 424/863] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:32.988812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:33.031320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34d13f-e479-454d-b4d8-56418090fad0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.067186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.067813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.068110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.068383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.069233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.070733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.084791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:33.093619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 440/864] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:32 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:33.093619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:33.140147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e70a18-643c-46a4-a014-a668de670922 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.202106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.202801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.203325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.203597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.215142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.221584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:33.232990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:33.234046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 22:30:33.234999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:33.235591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88554f-0692-4e6b-b7e9-6a5b92891060 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.241585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 425/865] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:30:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:33.241892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:33.268926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:33.279541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.280739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.282490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.282490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.282712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:33.283013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.284773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.295103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 441/866] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:33.370655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901c32-11bf-4c5d-9a63-23a23b0398ad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.408765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.409846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.410350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.410849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.411869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:33.412393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.414203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.417657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:33.435387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 426/867] 10.222.0.47 () {56 vars in 1135 bytes} [Mon Aug 1 22:30:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:33.443648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:33.555353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556718-d9b2-4e38-807e-62149ba0072a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.593253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.593949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.594232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.594504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.657830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:33.666746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.668224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.776436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 442/868] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:33 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:30:33.781882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:33.890196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2af96d-b519-4969-9caa-71708398658f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:33.925427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:33.926028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:33.926303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:33.926571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:33.936874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:33.938341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:33.953259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:33.954247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:33.955144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:33.955144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:33.955144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:33.955144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:33.957724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 427/869] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:33.960549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:34.068658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:34.079579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494fcf0-dda1-48a4-b010-c62b01745fb2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:34.123381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:34.124022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.129064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.129064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.129064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:34.129064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:34.129064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:34.139578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 443/870] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:34.145804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:34.198278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:34.254561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a9bad-0175-448f-b71a-fd7bb647f073 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:34.289097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:34.289769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.290046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.290312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.297360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:34.298863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:34.310541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:34.311468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:34.312366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:34.316547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 428/871] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:34.317983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:34.430636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4edc-78c3-4d5b-a55e-c7695908e3f0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:34.465393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:34.466055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.466332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.466607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.467393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:34.467715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:34.473595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb080303-f8f5-4673-9326-7fbebf3e1f28 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:34.486305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 444/872] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:34.492292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:34.597611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1671ec-ae0c-4005-9f18-7bce0014459b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:34.606956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:34.636156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:34.636780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.637052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.637317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.638140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=223d4641075e4b6489e3ee1e4fbdce36) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:34.638440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:34.639877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:34.648386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd5f4-43b3-4243-a494-5187a8e91482 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.648754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd5f4-43b3-4243-a494-5187a8e91482 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.649168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd5f4-43b3-4243-a494-5187a8e91482 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.650946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 445/874] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:34.651106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 429/874] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:34 2022] GET /identity/v3/role_assignments?user.id=223d4641075e4b6489e3ee1e4fbdce36&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:34.658321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929916f-72c6-4327-ab30-12f636947925 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.658931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929916f-72c6-4327-ab30-12f636947925 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.659450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929916f-72c6-4327-ab30-12f636947925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.684396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0929916f-72c6-4327-ab30-12f636947925 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:34.685754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0929916f-72c6-4327-ab30-12f636947925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:34.690148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:34.765618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 446/875] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:34.788698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:34.896968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d70f4-061f-46ff-bcb6-1361326ae994 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:34.931876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:34.932429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:34.932704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:34.932971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:34.945584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:34.951402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:34.952330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:34.953187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 430/876] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:30:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:34.960495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929916f-72c6-4327-ab30-12f636947925 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.067987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929916f-72c6-4327-ab30-12f636947925 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.103285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.104279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.104768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.105262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.106232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:35.106748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.108325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a11f2d-7a06-42d2-8f96-365367e9db16 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.117449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 447/877] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:30:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:35.123463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.161116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.237790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884099-5a0e-4fef-b49f-2082056408da None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.277064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.277064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.277064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.277064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.285855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.287325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.292698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ad6ef-a40b-47cb-ba01-072234e07440 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.293242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ad6ef-a40b-47cb-ba01-072234e07440 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.293756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ad6ef-a40b-47cb-ba01-072234e07440 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.295646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 448/878] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:35.301001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b76-f5e0-4839-a28f-e29129f33ff3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.301535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b76-f5e0-4839-a28f-e29129f33ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.302060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b76-f5e0-4839-a28f-e29129f33ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:35.304140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:35.304999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:35.305943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:35.305943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:35.305943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:35.305943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:35.305943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 431/879] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:35.312732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.316030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.329842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f44d7b76-f5e0-4839-a28f-e29129f33ff3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:35.331589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f44d7b76-f5e0-4839-a28f-e29129f33ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:35.404240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.421733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 449/880] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:35.421733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd53f5-e78f-4686-a595-6fea88acc928 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.421733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd53f5-e78f-4686-a595-6fea88acc928 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.421733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd53f5-e78f-4686-a595-6fea88acc928 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.422376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 450/881] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:35.427060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.427333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.427598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.431932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8b175-11dc-41cb-bf53-16d7e43fc394 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.451600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:35.452897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:35.467488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.468333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.468445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.468656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.469352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:35.469693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.471552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.488421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 432/882] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:35.488421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.529513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 451/883] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:35.530132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.591903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.594632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c85b4b-68e7-4c46-ba3d-9d2ffc19dae5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.630053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.630729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.631009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.631276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.631898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.633266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.634232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88625c-b39e-4372-8dc6-3bca36c8dc9d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:35.642123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:35.643342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:35.644437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:35.648150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 433/884] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:35.649161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.658128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.676309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.677307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.677789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.678221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.686726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.688174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:35.700209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:35.701227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 452/885] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:35.702237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.766474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd870-9898-4cad-9fb0-91cff27f535c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.801765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.802309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.802589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.802861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.803674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.805178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.813023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52008f8-cf9e-4a70-b95f-6db56559dc34 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:35.829660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 434/886] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:35 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:35.829660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.846257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.866102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:35.867065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:35.867485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:35.867927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:35.868761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:35.869239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:35.871073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:35.881403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 453/887] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:35.882978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:35.922767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:35.969944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840178b0-eb91-4b60-b613-c42295185feb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.012906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.013811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b086-76ba-43bf-a0a6-279087b4250c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.014574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.015049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.015591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.016858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:36.017370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.019055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.044600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 435/888] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:36.054250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.075604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.076854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.077359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.077971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.079331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:36.079849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.081996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.090783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 454/889] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:36.097234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.168062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:36.204787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd5315-177b-4493-aa8a-cf68cdce2a7f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.240735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:36.256623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.257641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.257987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.258460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.291547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7c36e-1e17-4203-8075-ee7518468fcd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.358388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.360770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.361270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.361702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.362622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] RBAC: Adding query filter params (interface=public, service_id=2af0603bfa234f4293fd64da75d1110a, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:36.363279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.366040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.384971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 455/890] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/endpoints?service_id=2af0603bfa234f4293fd64da75d1110a&interface=public®ion_id=RegionOne => generated 194 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:36.387428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.401930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.403671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.553147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 436/891] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 22:30:36.563451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.591207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4785-a0b6-4b71-9c1b-245885a730a6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.609043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:36.619054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:36.667985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.668830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.669546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.669949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.670859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.672469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a955fd-e4ee-49f0-afde-a61ce909b0b1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.689588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 456/892] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/services => generated 1268 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:30:36.732740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a51f-bce3-4993-8689-a81e7b321ac1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:36.765592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f6e8-f36e-4673-bf01-5a4fa5dc0dcc None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.765592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f6e8-f36e-4673-bf01-5a4fa5dc0dcc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.765872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f6e8-f36e-4673-bf01-5a4fa5dc0dcc None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.767408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 457/893] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:36.773213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.773539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.773858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.794722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:36.795421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:36.795700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:36.795975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:36.801962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:36.803529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:36.816781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:36.818625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:36.839542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:36.840545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:36.841451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:36.841451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:36.841451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:36.841451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:36.841451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 437/894] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:36.854743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.926635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 458/895] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:36.945501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:36.956692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:36.988746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aba23-e0cf-47eb-aa37-f4f61043604a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.059536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.060595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.061004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.061406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.062324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:37.063021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.064868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.080894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 438/896] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:37.087512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:37.170379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e243e-32b3-41da-959e-2b266bd8e83a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.175641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:37.209295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143beaf3-baba-4dcb-b66b-27f6aa654056 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.224433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.225100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.225367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.226067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.226449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.227825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:37.243974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:37.245070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:37.246338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:37.247099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 459/897] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:37.253414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:37.281163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.281864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.282147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.282488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.287648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:37.294415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.307093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:37.348503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:37.349467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:37.350399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 439/898] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:37.359882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:37.388028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55667d25-1c9b-4348-964f-ecde3a58f5a4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.425292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.426306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.426801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.427305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.428267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:37.428762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.430517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.436213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 460/899] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:37.441812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:37.480031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:37.561772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627938e8-c60a-46f6-9824-0d8fa421cfbb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.574968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9b15-4327-42c4-8297-02bc6ee814ab None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.603792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.604718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.605024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.605301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.606264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:37.606700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.608194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.621246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.622210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.622687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.623173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.624054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:37.624550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.626202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335b1d0-bc5d-450c-a6c9-93e7260e1f2f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.628415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 440/900] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:37.631965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:37.642226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 461/901] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:37.696949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:37.748046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f769d-06b7-4747-bf38-e966c20fbe39 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:37.792896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:37.794031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:37.794537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:37.795025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:37.798581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] RBAC: Adding query filter params (role.id=cfb5b6a1151a413abdb30900a49c3aee, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=feed14a81a6146338abb5ed7f7726e6c) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:37.798581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:37.798581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a46169-b2eb-4b66-b78c-d21613ab2ccc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:37.807509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 441/902] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:37 2022] GET /identity/v3/role_assignments?user.id=feed14a81a6146338abb5ed7f7726e6c&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=cfb5b6a1151a413abdb30900a49c3aee => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:38.093148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79518-46fb-4065-be0a-f671ad4b7dbf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.093148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79518-46fb-4065-be0a-f671ad4b7dbf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.093148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d79518-46fb-4065-be0a-f671ad4b7dbf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.093148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 462/903] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:38.099024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.099313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.099592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.128150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:38.129726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:38.216328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 442/904] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:38.251155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d79518-46fb-4065-be0a-f671ad4b7dbf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:38.365161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:38.372749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d79518-46fb-4065-be0a-f671ad4b7dbf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:38.415383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:38.415624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.415927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.416029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.416633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:38.418232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:38.427555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:38.428517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:38.429371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 463/905] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Aug 1 22:30:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:38.440907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:38.500355 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:38.553888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992de11-b5c3-4b24-948f-4b8ca5436283 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:38.590260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:38.591248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.591755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.592218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.593098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:38.593627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:38.595277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:38.601214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 443/906] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:38.606842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:38.749739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735f79-6167-40d4-8450-02c43b30c6ae None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:38.801194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:38.801883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.802164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.802489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.803184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:38.809424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:38.809424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dfa6a-9741-40e9-9b61-86af4beee946 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:38.813196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 464/907] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:38.819442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:38.930964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37c69-35f7-49a7-9760-3b7d1e43c96b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:38.950827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:38.970307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:38.971316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.971591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.971863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.972639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] RBAC: Adding query filter params (interface=public, service_id=8daf6d851cea414988069bd0838d9f50, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:38.972949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:38.974647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:38.976338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f715cc-cb0a-456f-9a57-cd8c8d1bb296 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.976620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f715cc-cb0a-456f-9a57-cd8c8d1bb296 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.976889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f715cc-cb0a-456f-9a57-cd8c8d1bb296 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.978544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 465/908] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:38.985602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:38.985602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:38.985602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:38.985961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 444/909] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:38 2022] GET /identity/v3/endpoints?service_id=8daf6d851cea414988069bd0838d9f50&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:38.990102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:39.017993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:39.020209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:39.049090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:39.127672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ede19-61ec-41c7-b265-3188dbebd21b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:39.156522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 466/910] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:39.177704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:39.177704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.177704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.177704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.178120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:39.178120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:39.184953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:39.185098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 445/911] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:38 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:30:39.318601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa342637-74f7-4a06-90a9-7c8dbf46ef27 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:39.363054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:39.364138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.364634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.365155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.366061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:39.367681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:39.378305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:39.379354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 22:30:39.380211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:39.388000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 467/912] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:39.390479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:39.453089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:39.517633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd5c6-9f3f-43ce-b6ee-8466c690fdde None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:39.560436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:39.561079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.561360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.561699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.562399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:39.562704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:39.564313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:39.570793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 446/913] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:39.577842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:39.609285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:39.753583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904656b-1939-4548-9d93-ec85517f50f5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:39.789598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:39.789598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.789598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.789598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.790018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:39.790018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:39.793581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3850f0a-287b-4b01-beeb-d0d408344c36 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:39.797284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 468/914] 10.222.0.47 () {56 vars in 1114 bytes} [Mon Aug 1 22:30:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:39.805385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:39.946307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fc816-24dd-4eeb-9b9b-d0309f4a7615 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:39.961947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:39.972508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8a9a8-b08c-43a2-9d54-edb3b0544aaa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.972811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8a9a8-b08c-43a2-9d54-edb3b0544aaa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.973035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8a9a8-b08c-43a2-9d54-edb3b0544aaa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.974436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 469/915] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:39.982720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.983002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.983268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.997613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:39.997613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:39.997613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:39.997613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:39.998022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:39.998022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900df8a8-1d40-4f55-9671-07b9e62460bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:40.012200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:40.017014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:40.033583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 447/916] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:40.097604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799356f-24b1-4159-b70f-6eb4566f96e8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.097604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799356f-24b1-4159-b70f-6eb4566f96e8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.097604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799356f-24b1-4159-b70f-6eb4566f96e8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.098116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 448/917] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:40.109157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.109157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.109157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.137644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:40.140581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:40.172528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 470/918] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:40.215569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:40.232234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:40.264268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 449/919] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:40.288023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:40.349794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735627ef-0a8e-4928-b817-2a89346e33c7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:40.390654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:40.390898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:40.391758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.397583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.397583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.397583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:40.397583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:40.402414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ab084-98c4-4a81-8d65-733703fcff45 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:40.425597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 471/920] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:40 2022] POST /identity/v3/services => generated 241 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:40.514599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:40.515577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.515929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.517589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.530056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:40.536450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:40.537957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:40.539316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 450/921] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:30:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:40.546757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:40.672483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:40.721514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d7359-ab21-4f11-8469-c0ec283b6769 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:40.758666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:40.758666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.758976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.759209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.759837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:40.760138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:40.765632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:40.771868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 472/922] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:30:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:40.778551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:40.888495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27daf-0674-401f-81bf-24209c32a3d5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:40.924999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:40.925681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:40.925963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:40.927970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:40.937259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:40.939098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:40.948693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:40.956093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:40.957017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:40.957993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:40.957993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:40.957993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:40.957993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:40.957993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 451/923] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:40.966779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.088686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e67-d00e-408f-8200-ecb2253f5467 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:41.122012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:41.128059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:41.128809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.129089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.129357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.130103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:41.130404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:41.133623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:41.149581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 473/924] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:41.152795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.279626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e75bf-a3bb-4625-8a63-5dbb1bb24f15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:41.319044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:41.319698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.319975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.320242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.328453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:41.330051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:41.342272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:41.343283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:41.344222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 452/925] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:41.350175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.460519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41559ec4-c5fa-4437-858c-82edc2ceeb86 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:41.503534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:41.505584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.505584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.505584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.505584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:41.505909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:41.510568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1adb0-15fd-45a2-8e21-80919f3b4cfa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:41.541834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 474/926] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:41.549944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.589153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e91516-0d1f-4917-a63f-d260c47ac876 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.589477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e91516-0d1f-4917-a63f-d260c47ac876 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.589739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e91516-0d1f-4917-a63f-d260c47ac876 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.591350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 475/927] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:41.596608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.596890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.597161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.617886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:41.618902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:41.652475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:41.689602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:41.724729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1902d7d-3b3c-413a-9794-2cfdc2444554 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:41.735121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 476/928] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:41.758967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.773282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:41.775743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.776494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.776971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.803767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:41.805213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:41.824190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:41.825160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:41.826109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:41.826109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:41.826109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:41.826109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:41.831023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 453/929] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:41.831696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:41.904653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8d8b2-4192-4259-a9f5-282a4855c3ce None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:41.954042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:41.955797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:41.956438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:41.957811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:41.959394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:41.960995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:41.980634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:41.981698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 22:30:41.982575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:41.993372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 477/930] 10.222.0.47 () {56 vars in 1132 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:41.994459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.010935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c742d-4771-4dcc-8568-9e48c61b546d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.031104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.059977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.061402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.062371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.062751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.063752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:42.066221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.068320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.086474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.099095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 454/931] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:42.100059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.172487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfdfc7-7838-43fb-be84-ea1511101278 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.223720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.224363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.224641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.224915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.225651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:42.225961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.227366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.248688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 478/932] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:42.254806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.298669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680c3e-6383-49ea-866e-31756bcd9b23 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.356178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.357520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.358060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.358529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.369350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.370912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bd7f9b6a-e817-4c11-9d64-77c541770f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:42.385980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:42.386940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:42.387837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:42.390535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 455/933] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:42.392054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbb29-f6df-4479-9144-76ae5e64cc47 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.392538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbb29-f6df-4479-9144-76ae5e64cc47 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.393024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbb29-f6df-4479-9144-76ae5e64cc47 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.398871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 456/934] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:42.399955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cbb29-f6df-4479-9144-76ae5e64cc47 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.414283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf78-2132-4a48-8c80-b425740faef1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.421620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.435424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.452740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.453371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.453676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.453950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.454677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:42.454985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.456574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2872d-98cb-4f56-85e7-2a90c3098b39 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.467081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 479/935] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:42.468457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.468784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.469008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.489403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:42.490645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:42.523195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cbb29-f6df-4479-9144-76ae5e64cc47 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.569320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.570040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.570318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.570587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.571338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:42.571641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.573069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.574916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 480/936] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:42.574916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.591247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 457/937] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:42.592197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.714115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.717237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:42.720461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8094942-27bb-436c-a640-777c9fbfa043 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.725040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d47d82-5971-4cdd-8201-3b41823c2f34 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:42.758495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.759841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.759980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:42.760118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.760390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.760517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.760777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.761049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/feed14a81a6146338abb5ed7f7726e6c/roles/cfb5b6a1151a413abdb30900a49c3aee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.761148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.762563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e43e5-ccec-48cd-8916-96357cfbb7f0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.763248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=feed14a81a6146338abb5ed7f7726e6c, role_id=cfb5b6a1151a413abdb30900a49c3aee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:42.764671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:42.782379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 458/938] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:30:42 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/feed14a81a6146338abb5ed7f7726e6c/roles/cfb5b6a1151a413abdb30900a49c3aee => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:30:42.782975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:42.789039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 481/939] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:42.790974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55bbe5-347a-40c9-b6e6-9d0826f5dbc9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.791300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55bbe5-347a-40c9-b6e6-9d0826f5dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.791578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55bbe5-347a-40c9-b6e6-9d0826f5dbc9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.793313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 482/940] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:42.799592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:42.799961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:42.800127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:42.822743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:42.824018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:42.973653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 483/941] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:42.975521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84785b-b863-40aa-a415-e2493ecf591f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.007404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.034945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.035497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.035765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.036004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.036694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.038783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.040932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING keystone.server.flask.common [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 22:30:43.073800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 22:30:43.073800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 22:30:43.073800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [parameters: {'id': '9b6eb9ab0cf847419c96d4a83affcfb5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 22:30:43.073800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b6eb9ab0cf847419c96d4a83affcfb5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b6eb9ab0cf847419c96d4a83affcfb5. Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:30:43.082346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 22:30:43.083205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Aug 01 22:30:43.084200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 22:30:43.085088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 22:30:43.086055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:30:43.086888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': '9b6eb9ab0cf847419c96d4a83affcfb5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.087748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:43.088613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b6eb9ab0cf847419c96d4a83affcfb5. Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.092593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 459/942] 10.222.0.47 () {60 vars in 1140 bytes} [Mon Aug 1 22:30:42 2022] POST /identity/v3/projects => generated 262 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:43.094357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.130597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:43.165465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:43.207697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6d669-e829-4b96-919c-9d44f9c799e5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.235388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bb5a4-227b-456b-96a0-ceddaa51154c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.278473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.281367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.281969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.282450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.283451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.284748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.285750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.286253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.286718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.288104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.299632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:43.305381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:43.306478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:43.307413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:43.308893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.311597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 484/943] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:43.316491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:43.326528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:43.327468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:43.328335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:43.329494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 460/944] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:43.335288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.422293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:43.464629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef93032-d2a8-4b36-a278-3ff989f61b45 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.485603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1239-39c3-46c0-98dd-1593b479318d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.504352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.504981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.505260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.505533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.506283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:43.506587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.508327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.542117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 461/945] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:43.551359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.552407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.552889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.553373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.554380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:43.554919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.556549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.566558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 485/946] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:43.577613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.697789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4935031-5e39-4af4-8bec-a210419ed374 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.729351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:43.745207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.746247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.746247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.746439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.747012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:43.747314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.748706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.755489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 462/947] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:43.762314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:43.778171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:43.871424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc042ee-cf78-49a1-9d91-085f7d6f4753 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:43.908918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:43.909671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:43.909946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:43.910212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:43.910974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] RBAC: Adding query filter params (interface=public, service_id=488f8b6374414849930d8f5ebed751d5, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:43.911277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:43.912637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d902-cc0d-401c-9077-44e13bc8a470 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:43.919675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 486/948] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/endpoints?service_id=488f8b6374414849930d8f5ebed751d5&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:43.927879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:44.057584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cd62-25fc-40e9-8992-59175b87f8be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:44.104266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:44.104910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:44.105197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:44.105469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:44.106193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:44.107576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51253f28-7f0e-4bc6-bad4-c49453887520 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:44.114838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 463/949] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:43 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:30:45.024770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9067-f2c2-4bd7-894d-dbeabb82ab13 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.025287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9067-f2c2-4bd7-894d-dbeabb82ab13 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.025287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9067-f2c2-4bd7-894d-dbeabb82ab13 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.026869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 487/950] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:45.032583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.032914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.033132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.055141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:45.056988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:45.065101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7738b0-e29f-4a7e-a065-fa6e97db6318 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.065392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7738b0-e29f-4a7e-a065-fa6e97db6318 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.065685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7738b0-e29f-4a7e-a065-fa6e97db6318 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.067262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 488/951] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:45.072282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.072558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.072827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.101716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:45.102910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:45.151732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:45.185753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 464/952] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:45.220632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.255236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 489/953] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:45.295979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.299809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:45.363596 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d678b84-436b-4498-9a22-03e51fc2977f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:45.402997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:45.404040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.404527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.405008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.417800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:45.423208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:45.424181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:45.425147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:45.432967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 465/954] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:45.434218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.455268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cfd74-542d-42cf-9223-282ca1930dcb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:45.493114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:45.519733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:45.520399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.520671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.520940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.521602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:45.522972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:45.534791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:45.535933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:45.536861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 490/955] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:45.551625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.595892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf66-08ca-402f-ae17-5eb71acb6c52 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:45.670439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:45.671073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.671348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.671614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.672345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:45.672640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:45.674137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:45.691377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:45.705263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 466/956] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:45.707895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.765758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d491d5f-63a3-442c-a910-493c9b0f72d7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:45.840447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:45.840982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.841251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.841522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:45.842245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:45.842540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:45.848221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:45.858573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 491/957] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:45.867963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:45.922712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438a844-4fc4-40d3-92b7-b9ae6951c44f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:45.939779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:45.986308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:45.987526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:45.987922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:45.988368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.005315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.007911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e9e28512-5aa0-49d9-90b8-c418027e7004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:46.029532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:46.030512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:46.031504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:46.031504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:46.031504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:46.031504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:46.034906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 467/958] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:46.036169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b84b-6207-44f0-99a0-3ff82663d520 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.036451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b84b-6207-44f0-99a0-3ff82663d520 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.036722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b84b-6207-44f0-99a0-3ff82663d520 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.038061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 468/959] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:46.040278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b84b-6207-44f0-99a0-3ff82663d520 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.047788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffda7a-23e1-429a-890f-1f2091e8af48 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.078888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.086534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.087472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.087939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.088421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.089738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:46.090354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.092234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6e9c-c97c-467e-b96e-0d55c2f107b0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.105028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 492/960] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:46.105028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61631f-5288-4892-86f5-c1074fc6dc58 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.105028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61631f-5288-4892-86f5-c1074fc6dc58 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.105028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61631f-5288-4892-86f5-c1074fc6dc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.135052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7a61631f-5288-4892-86f5-c1074fc6dc58 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:46.137060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7a61631f-5288-4892-86f5-c1074fc6dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:46.185886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b84b-6207-44f0-99a0-3ff82663d520 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.232872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 493/961] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:46.262943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.263956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.264455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.264939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.267198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad53e5-a33f-4303-bf77-fa1c20fbf756 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.267725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad53e5-a33f-4303-bf77-fa1c20fbf756 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.268190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad53e5-a33f-4303-bf77-fa1c20fbf756 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.268827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:46.269144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.270778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 494/962] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:46.273298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608aa84-157d-4c62-b3a5-dd5be54f0fe2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.273473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad53e5-a33f-4303-bf77-fa1c20fbf756 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.286730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 469/963] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:46.288468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.289026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.289447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.312153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:46.313668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:46.331049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.404196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 470/964] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:46.404918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.432045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad53e5-a33f-4303-bf77-fa1c20fbf756 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.465165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.476292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.476926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.477204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.477472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.478191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.479647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.482349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING keystone.server.flask.common [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 22:30:46.495940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.sql.core [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 22:30:46.495940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 22:30:46.495940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [parameters: {'user_id': '610c185896904a879fee01b7660a9393', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 22:30:46.495940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Aug 01 22:30:46.512644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:30:46.513983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application yield resource Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise self.value Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application session.commit() Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 22:30:46.514967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.flush() Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application compat.raise_( Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 22:30:46.515878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 22:30:46.516842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 22:30:46.519110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [parameters: {'user_id': '610c185896904a879fee01b7660a9393', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:46.521135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 22:30:46.523171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.525305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 495/965] 10.222.0.47 () {60 vars in 1131 bytes} [Mon Aug 1 22:30:46 2022] POST /identity/v3/users => generated 160 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:46.526927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a847b8-887a-492b-92bc-d69ab63b542d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.527073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.551665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.552319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.552595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.552866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.561761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.564119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:46.583933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:46.585157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 22:30:46.586324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:46.587543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 471/966] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:46.588480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.604710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.660123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52f6a-275a-47d8-86f5-723de3145770 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.710064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac240126-1520-42b7-9d03-00d30551a783 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.710490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.711305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.711856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.712131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.712929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.714949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:46.727621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:46.728696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:46.729578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:46.734293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 496/967] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:46.735244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.755159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.756202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.756629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.757151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.765886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.767470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:46.780320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:46.781321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 472/968] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:46.782616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.804554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.872954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:46.877469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555142-ba4a-45aa-b9cc-8bbb71ea32a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.908329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6a0b0-5412-45c5-a640-1b5be22dc95a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:46.916417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.916993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.917269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.917540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.918369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:46.918895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.920345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.958336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 497/969] 10.222.0.47 () {56 vars in 1149 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:46.959152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:46.960133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:46.961178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:46.961711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:46.962198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:46.963146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:46.963896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:46.965654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:46.973663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 473/970] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:46.973663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.104369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217e29f-f23b-4095-8d27-f2543b540915 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.109889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad334-9556-4245-a608-f7b485c6f7d1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.121442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.148974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.149959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.150242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.150608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.151380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:47.151678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.152548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.153181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.153309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.153475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.153761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.156547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.192337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 498/971] 10.222.0.47 () {56 vars in 1101 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:47.193118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.260407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.261928 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.365817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d914d8-bc70-422d-9228-5fba253e4299 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.415428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 474/972] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:30:47.422575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.425347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.426036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.426368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.426658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.427339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:47.427645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.429019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.449865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 499/973] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:47.453753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.472387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.502351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.571486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152c9d2-8c89-4100-8062-a8bd1343b9de None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.604532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac795ac-761d-43ab-8882-d81e277250cd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.616230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.616885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.617165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.617434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.629368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.631159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.644984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.645990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.646474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.646954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.647883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:47.648490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:47.649353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:47.650429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:47.650429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:47.650429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:47.650429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:47.652110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.654573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 475/974] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:47.655258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.686336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 500/975] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:47.777394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.795922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89266-4f03-4be4-86a9-c28fdaf91e6f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:47.843971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:47.844614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:47.844942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:47.845201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:47.846955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:47.847259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:47.848666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:47.865607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 476/976] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:47.871056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:47.973589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:47.989581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ced534-7284-4e79-974a-56f7bead5d14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:48.031414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:48.032074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.032449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.037587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.040358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:48.041890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-60746ae3-55b2-4f5c-8409-3e31b4553123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:48.057595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:48.058537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:48.059445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:48.062321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 501/977] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:48.063886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:48.186473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180f7f-7a78-4a87-bddb-606d2e3de785 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.186997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180f7f-7a78-4a87-bddb-606d2e3de785 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.189588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180f7f-7a78-4a87-bddb-606d2e3de785 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.190622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 502/978] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:48.195400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.195758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.195943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.208343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10d8e8-228b-44af-9357-aa6637226cea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:48.218718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:48.220353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:48.253778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:48.253778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.253778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.253778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.254376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:48.254376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:48.254847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:48.269614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 477/979] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:48.281661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:48.288057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:48.317801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 503/980] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:48.336349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:48.398812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93af70-bd90-4a4f-92b8-ff27f5271d34 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:48.425526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:48.432542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:48.433172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.433448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.433801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.434601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] RBAC: Adding query filter params (role.id=cfb5b6a1151a413abdb30900a49c3aee, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=feed14a81a6146338abb5ed7f7726e6c) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:48.434906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:48.436348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05cde7-c00e-4fc0-a46d-7427c0a3816b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:48.454021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 478/981] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:48 2022] GET /identity/v3/role_assignments?user.id=feed14a81a6146338abb5ed7f7726e6c&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=cfb5b6a1151a413abdb30900a49c3aee => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:48.454794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f8295-920c-4b6f-bb49-1e509af98e1b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:48.504094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:48.505932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:48.506547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:48.507030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:48.508798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:48.510469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fd44a-9777-47b1-80de-c7af1b5a3389 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:48.545160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 504/982] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:48 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:49.772419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ef3a6-6a20-4647-b3a0-0ac2a2ad5f39 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:49.773033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ef3a6-6a20-4647-b3a0-0ac2a2ad5f39 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:49.773033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ef3a6-6a20-4647-b3a0-0ac2a2ad5f39 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:49.774359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 479/983] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:49.779904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:49.780344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:49.780423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:49.799246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:49.800444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:49.878769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 505/984] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:49.901263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ef3a6-6a20-4647-b3a0-0ac2a2ad5f39 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:49.968757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.013397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ef3a6-6a20-4647-b3a0-0ac2a2ad5f39 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.049178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.049785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.050057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.050383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.051017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.052409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:50.061435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:50.062466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:50.068786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 480/985] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:50.071124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.136103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.177430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5f972-a43e-49a2-9162-95ede4904d71 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.212844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.213475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.213785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.214057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.214740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:50.215047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.216419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.222184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 506/986] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:50.227871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.331858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4e3a2-fea4-48dd-91df-09125cf040be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.361372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.369855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.370485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.370768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.371045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.371760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:50.372069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.373494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af69d-2d28-4b07-82bc-2383d8488eb1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.384789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 481/987] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:50.386097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.510223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34e1be-9b8d-461a-8e16-e6ea514ebb98 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.538946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.548192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.548828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.549103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.549375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.550199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] RBAC: Adding query filter params (interface=admin, service_id=488f8b6374414849930d8f5ebed751d5, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:50.550550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.552000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.561869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 507/988] 10.222.0.47 () {56 vars in 1249 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/endpoints?service_id=488f8b6374414849930d8f5ebed751d5&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:50.563789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b40a-8207-403d-b3de-d095767fe082 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.564073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b40a-8207-403d-b3de-d095767fe082 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.564346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b40a-8207-403d-b3de-d095767fe082 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.566345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 482/989] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:50.566622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.572654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899888-012a-40e2-a3aa-26098f3fd0e0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.572934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899888-012a-40e2-a3aa-26098f3fd0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.573210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899888-012a-40e2-a3aa-26098f3fd0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.607801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d899888-012a-40e2-a3aa-26098f3fd0e0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:50.608917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d899888-012a-40e2-a3aa-26098f3fd0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:50.684029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95189e9f-321f-4b63-93fb-9a3ec7af1a9c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.696485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 483/990] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:50.697803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8abdf-0d15-4ece-9ca2-da904fd5cb73 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.698091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8abdf-0d15-4ece-9ca2-da904fd5cb73 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.698359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8abdf-0d15-4ece-9ca2-da904fd5cb73 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.699760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 484/991] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:50.707017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.707299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.707573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.729207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:50.730440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:50.734410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.735282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.735907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.736543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.737763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.739951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.747484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 508/992] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:30:50.748508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.819062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.820566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 485/993] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:50.844967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.868707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00205ad5-f876-4bf3-8cc2-34b9d8790189 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.904555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:50.907400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.908104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.908521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.908702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.909342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.910820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:50.920565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:50.921617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:50.922473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 509/994] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:50.928565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:50.957772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bd18e-1594-4bb1-a6c1-82e99100ec95 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:50.995117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:50.996006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:50.996233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:50.996516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:50.997142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:50.998576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:51.008361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:51.009425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:51.010338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 486/995] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:51.017262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:51.041228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc58a26-a69a-4530-a365-dde9ba9876a5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.061636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:51.077379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.078061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.078340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.078607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.079354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:51.079654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.081023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.085975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 510/996] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:51.092639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:51.131327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc5e9-435f-4e7c-b800-7990ca98ae30 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.152135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:51.172251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.172906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.173214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.173492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.174207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:51.174520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.175909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.185957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 487/997] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:51.187334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:51.207649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773acdea-b712-43d2-a530-7d9b1622eac4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.246464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.247144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.247447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.247718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.248562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] RBAC: Adding query filter params (type=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:51.248865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.250382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.255247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 511/998] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:51.302778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7baff6-f134-4657-8c5d-4f9280c55913 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.341826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.342491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.342767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.343037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.343848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:51.344150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.345766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.352084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 488/999] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:51.358156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:51.421384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:51.472937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57180ec3-fd92-4896-8962-19ba1cfac210 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.508076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.508698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.508976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.509247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.510043 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] RBAC: Adding query filter params (interface=public, service_id=da25cc2952854e38a1894584d457056f, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:51.510349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.511725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e48b8-4f7a-4600-af85-88e117341fa7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.517143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 512/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:51 2022] GET /identity/v3/endpoints?service_id=da25cc2952854e38a1894584d457056f&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:51.523228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:51.573979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:51.631048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27661a0d-1de5-4d42-aecc-9d4efafc97bd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:51.665838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:51.666558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:51.666837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:51.667114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:51.667804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:51.669185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992229ea-c858-4912-a167-f60f794afb36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:51.677685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 489/1001] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:51 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:30:52.759570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5ad4a-8466-40b0-9803-9c44a723fa7e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:52.760125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5ad4a-8466-40b0-9803-9c44a723fa7e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:52.760125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5ad4a-8466-40b0-9803-9c44a723fa7e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:52.761632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 513/1002] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:52.767529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:52.768164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:52.768164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:52.788592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:52.789841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:52.865594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 490/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:52.887570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5ad4a-8466-40b0-9803-9c44a723fa7e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:52.994938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5ad4a-8466-40b0-9803-9c44a723fa7e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.008895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:53.035250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.035892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.036164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.036433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.037054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.038447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:53.047637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:53.048722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 22:30:53.050022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.053896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 514/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Mon Aug 1 22:30:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:53.055137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:53.099563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:53.159419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599344-591b-4f56-b95b-45739c69e9b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.196293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.196753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.197095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.197374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.198145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:53.198454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.199849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069b318-b9e0-4359-afe3-e5ddfbaa28a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.209941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 491/1005] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:53.211113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:53.294621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4df7d-0f99-4911-86ec-5949612b6437 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.294927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4df7d-0f99-4911-86ec-5949612b6437 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.295125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4df7d-0f99-4911-86ec-5949612b6437 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.296617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 492/1006] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:53.301657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.301657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.301967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.321296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772494a-1fb5-4764-8171-c8e5aca8ad17 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.324538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:53.325937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:53.359754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.360696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.361155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.361625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.362453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:53.362973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.364480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.370154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 515/1007] 10.222.0.47 () {56 vars in 1122 bytes} [Mon Aug 1 22:30:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:53.375556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:53.402535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 493/1008] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:53.424381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:53.426690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:53.461021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:53.490368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725672f-b289-4021-a39e-28fb403fe491 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.528899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b40b0-868d-40c6-a40f-dd625a8e2f0c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.533597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.534543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.534820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.535088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.535793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.537602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fd3a1-9d55-4489-8bdd-64b7931410c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.569678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 516/1009] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:53.590046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.590707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.590987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.591258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.591974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.593403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b1e6-fa17-4d5e-8334-41b513f7bea4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.621633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 494/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:53 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:53.679599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371501-612c-4883-8fdb-abe3a451d423 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.679897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371501-612c-4883-8fdb-abe3a451d423 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.680144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a371501-612c-4883-8fdb-abe3a451d423 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.681796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 517/1011] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:53.687841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.688204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.688370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.708686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:53.709942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:53.797780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 495/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:53.820267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a371501-612c-4883-8fdb-abe3a451d423 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:53.922402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:53.927583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a371501-612c-4883-8fdb-abe3a451d423 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:53.966342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:53.966988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:53.967262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:53.967530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:53.968141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:53.969503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:53.979293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:53.980350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 22:30:53.981641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:53.985479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 518/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:53.987111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:54.009139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:54.101279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e82b3-a89c-47b2-a944-cf1105eb56f4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:54.136497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:54.137120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:54.137397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:54.137700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:54.138381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:54.138693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:54.140051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:54.145192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 496/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:54.150991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:54.257279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190dfa-dd51-4552-8128-3e31a2e4fc79 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:54.291904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:54.292924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:54.293412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:54.293933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:54.294817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:54.295333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:54.296948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a102ce-26eb-41bf-9d15-9feae247f77d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:54.302795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 519/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Mon Aug 1 22:30:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:54.308532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:54.400151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:54.423413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208661ef-9fc9-47c1-8b6b-f97203bb0ee4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:54.458189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:54.458852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:54.459125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:54.459396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:54.460131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:54.461496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1bb3-e790-4fef-ac77-35c04caa2db5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:54.481851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 497/1016] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:55.586261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f3e9e-24ca-4c38-b6d4-1490604c38bc None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.586803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f3e9e-24ca-4c38-b6d4-1490604c38bc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.586803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f3e9e-24ca-4c38-b6d4-1490604c38bc None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.588163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 520/1017] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:55.593873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.594318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.594514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.614064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:55.615268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:55.647406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8915-6edc-466a-bf1c-5b9a95b5a7bc None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.647950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8915-6edc-466a-bf1c-5b9a95b5a7bc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.648226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8915-6edc-466a-bf1c-5b9a95b5a7bc None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.649982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 521/1018] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:55.655406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.655672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.655966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.676431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:55.677658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:55.695984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 498/1019] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:55.702226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:55.719447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:55.751487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 522/1020] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:30:55.773634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:55.780184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:55.821536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8502d5e-9e82-473f-b1b0-815e02e39485 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:55.856263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:55.856836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.857115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.857385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.858558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:55.860008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:55.866458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 499/1021] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:55.867627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:55.878323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba215f48-b1d5-4818-a550-79c5b50c7f83 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:55.913096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:55.913988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:55.914520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:55.914948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:55.915761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:55.917256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:55.926959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:55.928202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:55.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:55.933178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 523/1022] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Aug 1 22:30:55 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:55.934234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:55.959743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:55.971103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed100-e4c7-4144-b8df-4f19905d064f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.002718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:56.009131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.012945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.012945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.012945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.012945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:56.014309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:56.046437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 500/1023] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:30:55 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:30:56.048351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389621bf-3dfe-450f-a002-b7c1e217730a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.086451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.087100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.087375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.087648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.088349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:56.088655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:56.090077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:56.095963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 524/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:56.102544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:56.216440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd471-24ec-4527-86f4-03b4261f7a2f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.252719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.253337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.253636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.253910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.254613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:56.254919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:56.256365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:56.262251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 501/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:56.268383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:56.362710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:56.379116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e129e60-bddd-4ecd-a902-ea60c631d848 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.413093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.413752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.414031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.414301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.415069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] RBAC: Adding query filter params (interface=public, service_id=91f9ce67b0b042c5b1f10d2470f1b079, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:56.415376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:56.416747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032cb09-26b6-4a3c-b94d-cf9ded0341b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:56.422783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 525/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:30:56 2022] GET /identity/v3/endpoints?service_id=91f9ce67b0b042c5b1f10d2470f1b079&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:56.428610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:56.479015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:56.534163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5613-58a3-45d2-827e-bafd9e53cdf9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.562743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b94-766e-42e0-af82-7e91b3896ca6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.563065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b94-766e-42e0-af82-7e91b3896ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.563290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b94-766e-42e0-af82-7e91b3896ca6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.565041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 526/1027] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:30:56.569379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.569836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.570061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.570163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.570360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.570477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.570668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.571328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:56.572688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:56.580685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 502/1028] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:30:56 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:30:56.589994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:56.591222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:56.669884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 527/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:56.693354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:56.805327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc692bb-110f-4551-8cde-e0f395a43c57 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:56.816832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:56.841971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:56.842626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:56.842901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:56.843167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:56.853856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:30:56.859369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:56.860297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:56.866353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 503/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:30:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:56.868848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:56.951731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:56.972987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0492d-b5fd-4f16-8a96-60b1ad82749a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.013938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.014572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.014895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.015110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.015851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:57.016153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.017666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:57.026462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 528/1031] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:30:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:57.032363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:57.139067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f01a3-abbf-4b5e-8aad-6c0e47a37176 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.173161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.173852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.174135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.174410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.185208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.186705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:57.202439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:57.203508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:57.204469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:57.204469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:57.204469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:57.204469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:57.204469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 504/1032] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:57.209675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:57.312527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912342dd-125b-4a2c-91a2-0c1d9ffc6518 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.347204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.347851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.348096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.348389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.349088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:57.349386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.350871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:57.373123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 529/1033] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:57.375825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:57.405859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:57.493301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113edeeb-cdcf-43ba-b8ed-d065847171af None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.529932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.530597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.530840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.531127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.540168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.541616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:30:57.553121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:57.554149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:30:57.555122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:30:57.557658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 505/1034] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:30:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:57.560407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:57.563394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:57.662084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4041-8304-4fa1-99c4-e43f4ee94cdf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.696694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.697338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.697631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.697908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.698749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:57.699053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.700541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:57.725194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 530/1035] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:30:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:57.726547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:57.813323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:57.832956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa6c7-6f28-4b96-835a-e98a5bc0c61a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:57.867548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:57.868167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:57.868446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:57.868718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:57.938019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:57.939445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d7017-e2b5-4797-9d5d-b333a861fcca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.028903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.053316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 506/1036] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:30:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:30:58.055032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.101499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343730d-c926-448b-ae32-cd24eda01628 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.101851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343730d-c926-448b-ae32-cd24eda01628 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.102092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343730d-c926-448b-ae32-cd24eda01628 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.105612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 507/1037] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:58.108708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.109289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.109604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.119415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.128630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:58.129842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:58.164565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8557-4a77-4270-b597-f245084eec56 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.198015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 508/1038] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:58.205212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.206079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.206529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.206953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.218052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.219741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.222539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:30:58.240401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:58.241578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:58.242720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:30:58.242720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:30:58.242720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:30:58.242720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:58.244245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 531/1039] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:58.249886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.324265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d5301-b741-4694-ad91-e25d92d5c841 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.338326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.365343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be39489-31d3-46e6-bff7-e7146b6dfa55 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.365548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.366126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.366409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.366674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.367885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.369389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.375902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 509/1040] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:58.377322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.379614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.401118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.401941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.402067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.402422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.403125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:58.403421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.404784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.421370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 532/1041] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:58.423125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.488110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3659e3-c957-4f05-9992-75691d8e578c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.524023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.524642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.524919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.525187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.525732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5813cf-4138-42eb-9e4a-a25ce4bd3087 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.525906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.527458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.561586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 510/1042] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:30:58 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:30:58.561800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.561800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.561800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.562111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.570414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.571918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bea52942-467c-44d2-bfeb-c06ef6d6bc5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:30:58.585313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:58.586427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:58.587368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 533/1043] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:30:58.592936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.596832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bb60-71bc-4aaa-b0c5-85be8bb22bad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.597116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bb60-71bc-4aaa-b0c5-85be8bb22bad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.597387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7bb60-71bc-4aaa-b0c5-85be8bb22bad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.599274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 534/1044] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:30:58.604663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.604940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.605209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.624126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:30:58.625370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:30:58.640145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.657008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.703963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 535/1045] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:30:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 22:30:58.718567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068de9-19ee-47ff-8441-09a157270c40 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.728550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.759738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.760283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.760553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.760821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.761614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:58.761920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.763369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.776853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 511/1046] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:30:58.783918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.842467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d8db3-64b7-4120-8b2f-8debf9089957 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.878877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.879519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.879797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.880054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.880693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.882099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:30:58.891191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:30:58.892190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:30:58.893046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 536/1047] 10.222.0.47 () {56 vars in 1105 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:30:58.898451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:58.898781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.907134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861e23-a1a5-4055-a6ed-7c801cd0cff0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:58.929535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:58.943726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:58.944699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:58.944699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:58.944929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:58.945690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] RBAC: Adding query filter params (role.id=cfb5b6a1151a413abdb30900a49c3aee, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:58.946000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:58.947437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:58.957754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 512/1048] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=cfb5b6a1151a413abdb30900a49c3aee => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:59.020223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b62f1-89e7-455e-adf9-1a4fbd5e4f69 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:59.059509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:59.060143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:59.060418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:59.060683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:59.061374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:59.061707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:59.063099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:59.069437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 537/1049] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:59.075306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:59.192000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4665-be4c-41e2-b246-83b8e79bf86b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:59.227132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:59.227740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:59.228021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:59.228293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:59.228972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:30:59.229273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:59.230694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53d54d-13aa-4a46-8b6c-c4a1154ae489 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:59.235786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 513/1050] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:30:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:30:59.241377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:30:59.334637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:30:59.346745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0c15a-0b26-4790-b738-907f2d23786b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:30:59.382018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:30:59.382924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:30:59.383355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:30:59.383773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:30:59.384641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:30:59.386165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0276684-9747-4abb-9579-c455e919eb1c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:30:59.410664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 538/1051] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:30:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:00.632519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a665b-1ec6-4332-b924-f2afc4b49ca4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:00.633085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a665b-1ec6-4332-b924-f2afc4b49ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:00.633085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a665b-1ec6-4332-b924-f2afc4b49ca4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:00.634623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 514/1052] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:00.639876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe2ac-e6cc-4a85-ac6e-c502362700cd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:00.640294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe2ac-e6cc-4a85-ac6e-c502362700cd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:00.640486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe2ac-e6cc-4a85-ac6e-c502362700cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:00.658974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b5bfe2ac-e6cc-4a85-ac6e-c502362700cd None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:00.660179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b5bfe2ac-e6cc-4a85-ac6e-c502362700cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:00.737886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 539/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:00.761154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a665b-1ec6-4332-b924-f2afc4b49ca4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:00.789848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:00.871983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a665b-1ec6-4332-b924-f2afc4b49ca4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:00.907943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:00.908513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:00.908789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:00.909063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:00.916197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad228-455c-47fa-99ff-083223aabdea None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:00.916471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad228-455c-47fa-99ff-083223aabdea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:00.916747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad228-455c-47fa-99ff-083223aabdea None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:00.918281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 540/1054] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:00.922416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:00.923825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:00.924101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:00.924374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:00.927487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:00.928410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:00.929273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 515/1055] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:00.934878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:00.948607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:00.949831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:01.025631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 541/1056] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:01.028141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.045810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565c55f-1120-4d23-9b93-a9eba00afb8e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.047388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.049494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.083859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.084486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.084780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.085023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.085778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:01.086077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.087457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.097359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 516/1057] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:31:01.102843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.165455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa80b98-fb29-46de-9c31-a18dfb413953 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.202709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.203302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb88b2-f9e2-4667-8156-a9a41e8e98a9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.203417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.203634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.203908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.215195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:01.220857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:01.222268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:01.223117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 542/1058] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:01.228437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.241514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.242513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.242943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.243332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] PATH_INFO: `/v3/users/42dba402ffea4c03929723898224b328` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.260885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] RBAC: Authorizing `identity:get_user(user_id=42dba402ffea4c03929723898224b328)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.262357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.264072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 517/1059] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/users/42dba402ffea4c03929723898224b328 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:31:01.269806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.273598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.297494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.339295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925237b2-1fa8-4655-a645-44254a98b37a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.374918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.375547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.375830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.376132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.376242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b59d95-b279-4f38-8ba0-455cf1bf7562 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.376861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:01.377165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.378591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.387250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 543/1060] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:31:01.392980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.416733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.417283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.417605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.417871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.485784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.487253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcc0da-3e78-4e8f-b439-c43787ab6f72 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.489838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.490646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85659e85-910d-4aca-92c5-9a335e2f6fff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.513586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.525183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.525855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.526125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.526390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.537773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.539217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7d7e9416-e368-4149-b67f-a60e00d28ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:01.553491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:01.554698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:01.555609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:31:01.555609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:01.555609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:01.555609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:01.555609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 544/1061] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:01.555933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac360f-7832-4392-bf17-e16ef9557648 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.556070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac360f-7832-4392-bf17-e16ef9557648 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.556340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac360f-7832-4392-bf17-e16ef9557648 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.557603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 545/1062] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:01.560318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac360f-7832-4392-bf17-e16ef9557648 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.590601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 518/1063] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 22:31:01.592042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.592303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.592578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.610190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:01.611449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:01.661468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac360f-7832-4392-bf17-e16ef9557648 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.677663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 519/1064] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:01.678246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.702766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.703648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.703930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.704224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.704921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:01.705224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.706650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.712123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.720961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 546/1065] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:31:01.722181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.762671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.782317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd78a-2eb2-466c-bb0b-4ec2d379ceea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.818174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.818814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.819091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.819361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.820544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.822018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.823703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 520/1066] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:01.824429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:01.826176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743330e6-0420-4958-adcc-d8ec0a7d101c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.859678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.860295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.860599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.860872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.923509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711fe57-f551-46b6-9953-35fdaef3d1b1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:01.925052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.928142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.929580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.957789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:01.958432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:01.958709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:01.958980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:01.961127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:01.966102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:01.967560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:31:01.979590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:01.980608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:01.981696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 521/1067] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:01.982731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.026079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 547/1068] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:02.026977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.088671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008099-6ca6-4dcb-b5b2-b964c39d913a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.121476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e973-37b8-4c6c-9fdf-f94e3ae0bf0d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.122142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.122736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.123047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.123316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.124102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=fca29081c46b4f309d8fd856d9bba519, user.id=42dba402ffea4c03929723898224b328) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:02.124405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.125874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.131692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 522/1069] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:01 2022] GET /identity/v3/role_assignments?user.id=42dba402ffea4c03929723898224b328&scope.project.id=fca29081c46b4f309d8fd856d9bba519&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:02.132371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.162473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.174050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.174787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.175138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.175480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.176368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:02.176746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.178397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.182630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.206940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 548/1070] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:02.208171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.257981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3551f-9fe4-4c77-a738-a447fcfec226 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.295625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.296285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.296563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.296832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.314288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb22ee-ae9a-4fc1-82f5-9acd3207633e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.314878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.316416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:02.331222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:02.332187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:02.333099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:31:02.333099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:02.333099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:02.333099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:02.333099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 523/1071] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:02.338338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.351489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.352141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.352419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.352688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.363001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.364439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:02.382176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:02.383160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:02.384065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:31:02.384065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:02.384065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:02.384065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:02.387887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 549/1072] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:31:02.389994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.426443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.442714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.450636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e35a1-902d-4666-af45-c6188c16ebb7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.486887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.487534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.487810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.488072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.488759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:02.489055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.490453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.498646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3b56c-b0c4-4aee-a2e5-3a16d178f12b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.502608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 524/1073] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:31:02.538025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.538673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.538952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.539221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.539942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:02.540251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.541689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.561652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 550/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:02.563761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.675918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103e474-5723-4feb-9838-47ff511df21d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.709919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.710640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.710842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.711115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.717998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.719551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.721830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:31:02.732743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:02.733741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:02.734660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 525/1075] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:02.740071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:02.845892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3529a-748e-40ab-baa1-4d90588298f5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:02.853739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:02.884388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:02.885014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:02.885316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:02.885616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:02.886371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:02.886676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:02.888112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924e23-3331-4cdd-a2cf-7cf08bf5c6bb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:02.900432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 551/1076] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:02.906446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:03.006857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c4834-dd7c-4782-a649-9ce3fe4bd7cb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:03.041066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:03.041765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:03.042041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:03.042312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/cfb5b6a1151a413abdb30900a49c3aee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:03.044520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=b74c1e8e8d3643db9230f448af705d3a, role_id=cfb5b6a1151a413abdb30900a49c3aee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:03.045999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40522d-4d84-4a6d-b59f-d7f23d9215ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:03.059583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 526/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:31:02 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/b74c1e8e8d3643db9230f448af705d3a/roles/cfb5b6a1151a413abdb30900a49c3aee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:31:04.120333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb898597-625b-454e-be3b-c952fea9a7d5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.120916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb898597-625b-454e-be3b-c952fea9a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.120916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb898597-625b-454e-be3b-c952fea9a7d5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.122260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 552/1078] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:04.127813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.128275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.128369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.148173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:04.149379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:04.204695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:04.220903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 527/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:04.243261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb898597-625b-454e-be3b-c952fea9a7d5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:04.349474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb898597-625b-454e-be3b-c952fea9a7d5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:04.382856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:04.383485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.383768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.384034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.400660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:04.405588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:04.406566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:04.407421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 553/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:04.413247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:04.514177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde69a97-babb-4df8-bd56-9d5de1f5c4b8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:04.549162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:04.549877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.550132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.550522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.551264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:04.551569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:04.552963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:04.565894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 528/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:04.567811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:04.600285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:04.686166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadefb-d39b-437c-aebd-a59546272b64 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:04.720763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:04.721405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.722082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.722347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] PATH_INFO: `/v3/users/42dba402ffea4c03929723898224b328` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.723501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=42dba402ffea4c03929723898224b328)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:04.724943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:04.726602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 554/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:04 2022] GET /identity/v3/users/42dba402ffea4c03929723898224b328 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:04.732639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:04.784091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:04.837349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28214a59-4578-4b23-8846-d3369ee5ef38 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:04.875407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:04.875969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:04.876330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:04.876604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:04.877798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:04.879270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc373-771d-4e20-9f1b-6dff73a63f0b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:04.880953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 529/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:04 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:04.887413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:04.985400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:04.992810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5512d2-28fa-43f5-9aec-f525a5a07ec7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.026773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8239df-0201-4bc3-b062-d309938a0802 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.027055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8239df-0201-4bc3-b062-d309938a0802 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.027324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8239df-0201-4bc3-b062-d309938a0802 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.028682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 530/1084] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:05.032228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.033146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.033427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.033670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.033771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519/users/42dba402ffea4c03929723898224b328/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.033953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.034222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.035724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fca29081c46b4f309d8fd856d9bba519, user_id=42dba402ffea4c03929723898224b328, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:05.037176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:05.051685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 555/1085] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:31:04 2022] PUT /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519/users/42dba402ffea4c03929723898224b328/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:31:05.054089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:05.055287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:05.132967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 531/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:05.156481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:05.268622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9621d-132d-436f-9a76-bbcabcd856b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.301736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.302379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.302651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.302941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.312975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:05.318507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:05.319447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:05.320539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 556/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:05.325948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:05.376212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:05.427000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcc9f2-2c73-49a2-933e-8d4ba9a2cada None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.459807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.460428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.460709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.461041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.461806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:05.462109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:05.463483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:05.472114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 532/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:05.477959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:05.494408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:05.583851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957bca-ad31-4a99-a211-b028df73c867 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.618562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.619199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.619474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.619745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.630482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:05.631937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:05.646166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:05.647139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:05.648603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:31:05.648603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:05.648603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:05.648603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:05.648603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 557/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:05.653183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:05.753811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8cf1-cf1a-472d-8b27-b67d1011d5ac None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.764118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:05.788524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.789149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.789426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.789727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.790411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:05.790710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:05.792079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:05.804002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 533/1090] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:05.809883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:05.876965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:05.911684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3e833-97f9-4a9d-9dac-9cd0f331d4db None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:05.945442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:05.946099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:05.946375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:05.946643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:05.955125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:05.956574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:31:05.967965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:05.968921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:05.969900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 558/1091] 10.222.0.47 () {56 vars in 1093 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:05.975371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:06.077355 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811141e-ca74-4835-9639-c82cde7466a0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:06.112665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:06.113295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:06.113594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:06.113865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:06.114659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:06.114958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:06.116383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:06.133289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 534/1092] 10.222.0.47 () {56 vars in 1133 bytes} [Mon Aug 1 22:31:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:06.139313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:06.241708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91bc65-1898-4ee0-ae2b-61f3ddb9cddb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:06.267077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:06.276083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:06.276692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:06.276970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:06.277234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:06.341036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:06.342498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:06.444590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 559/1093] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:06.450560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:06.480011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:06.553575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b4600-b571-4202-8ce1-a2c1f8b0c7c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:06.588730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:06.589387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:06.589703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:06.589983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:06.600976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:06.602465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:06.617334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:06.618393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:06.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:31:06.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:06.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:31:06.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:06.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 535/1094] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:06.624603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:06.652326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:06.728192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21128734-bdde-4a3d-9950-647a8832e823 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:06.764104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:06.764732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:06.765010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:06.765281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:06.765994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:06.766301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:06.767775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:06.779270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 560/1095] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:06.785087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:06.870552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:06.893654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855737d-8f03-4c89-8fb6-08a55c2eb6eb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:06.927337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:06.927975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:06.928250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:06.928515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:06.935743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:06.937273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4a043997-e8d0-4104-b619-458486e65ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:31:06.949048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:06.950077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:06.951008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 536/1096] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:06.956228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.029985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503af47-49a1-4d83-b9b4-43ac12c0db6c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.030272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503af47-49a1-4d83-b9b4-43ac12c0db6c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.030546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503af47-49a1-4d83-b9b4-43ac12c0db6c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.032025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 537/1097] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:07.037316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.037626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.037906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.046472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.056757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:07.057990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:07.060857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e598cf7-22aa-473e-a101-47726cf58d7f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.094701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.095375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.095638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.095903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.097013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:07.097310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:07.098775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:07.110680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 561/1098] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:07.116699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.125377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 538/1099] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:07.148645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.215835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.216687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220356-fc2d-42c0-a0a9-e241101cd36e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.248365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b63cb7-1dc5-45c2-8d8d-ee0c6535254c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.251989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.252602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.252876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.253145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.253985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] RBAC: Adding query filter params (role.id=cfb5b6a1151a413abdb30900a49c3aee, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=b74c1e8e8d3643db9230f448af705d3a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:07.254292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:07.255727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:07.261795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 562/1100] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/role_assignments?user.id=b74c1e8e8d3643db9230f448af705d3a&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=cfb5b6a1151a413abdb30900a49c3aee => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:07.288989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.289701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.289984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.290254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.305458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:07.310514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:07.311562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:07.312420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 539/1101] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:31:07.318523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.332769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.427828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ef68f-0dcc-4a34-88ff-f63facf1e8c9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.463583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.464211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.464485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.464758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.465473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:07.465829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:07.467190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:07.475708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 563/1102] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:07.481872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.581110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b2992-86ca-4d9b-ad50-d0abd33003dc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.610520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.617613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.618306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.618594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.618869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] PATH_INFO: `/v3/users/42dba402ffea4c03929723898224b328` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.620067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] RBAC: Authorizing `identity:get_user(user_id=42dba402ffea4c03929723898224b328)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:07.621522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:07.623211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 540/1103] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/users/42dba402ffea4c03929723898224b328 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:07.629359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:07.705357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.737906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70e8ed-5156-490b-8976-7b14ac67d51c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:07.776429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:07.777102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:07.777369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:07.777674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:07.845941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:07.847758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:07.915010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:07.946303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 564/1104] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:07.952672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:08.053379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddbdfb-78fa-4437-bb6b-913d30b35d8a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:08.086751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:08.087390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:08.087666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:08.087936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:08.089088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:08.090550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb332c-3ce0-489f-b9fd-84b289d9e76c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:08.092217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 541/1105] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:07 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:08.098891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:08.218600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918247b2-df09-41ea-9610-e1b2b89aca7d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:08.276459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:08.277416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:08.277509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:08.277881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:08.279182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=fca29081c46b4f309d8fd856d9bba519, user.id=42dba402ffea4c03929723898224b328) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:08.279699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:08.282159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77fac8-6238-4582-9df6-c1593d562a3e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:08.293222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 565/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:08 2022] GET /identity/v3/role_assignments?user.id=42dba402ffea4c03929723898224b328&scope.project.id=fca29081c46b4f309d8fd856d9bba519&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:10.328453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8d33-277e-45bf-819d-564cf5e80d6e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:10.329007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8d33-277e-45bf-819d-564cf5e80d6e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:10.329007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8d33-277e-45bf-819d-564cf5e80d6e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:10.330350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 542/1107] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:10.336631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:10.337353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:10.337534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:10.359406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:10.360695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:10.389901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:10.436562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 566/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:10.464250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d8d33-277e-45bf-819d-564cf5e80d6e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:10.535014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:10.570993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d8d33-277e-45bf-819d-564cf5e80d6e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:10.605909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:10.606479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:10.606760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:10.607025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:10.608201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:10.609665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a39c7-8a17-4bf2-8b6b-cc7055f589af None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:10.611309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 543/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:10.617407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:10.721265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88e1e-f6e7-4b3b-8b0a-74269eee7888 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:10.764469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:10.765077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:10.765351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:10.765647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:10.766318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:10.767781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520a24b-d5e2-4d8f-9d5e-db4c6dbb0f12 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:10.790520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 567/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:31:10 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:12.808171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c90793-0285-4dec-852e-3ecc08004bd6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:12.808775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c90793-0285-4dec-852e-3ecc08004bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:12.808775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c90793-0285-4dec-852e-3ecc08004bd6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:12.810114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 544/1111] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:12.816348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:12.817036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:12.817179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:12.840735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:12.841973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:12.859885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:12.912610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 568/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:12.943215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c90793-0285-4dec-852e-3ecc08004bd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:13.058335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c90793-0285-4dec-852e-3ecc08004bd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:13.092857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:13.093493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:13.093804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:13.094080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] PATH_INFO: `/v3/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:13.095704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] RBAC: Authorizing `identity:get_role(role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:13.097448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:13.098848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 545/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:12 2022] GET /identity/v3/roles/0faca02de5824306b70c798c90d5f033 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:13.105758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:13.224782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d294ed-25a8-4a99-aec1-20b3b155d17d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:13.260266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:13.260897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:13.261171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:13.261435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] PATH_INFO: `/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:13.274989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=cea1ee95447e4ba2b4a12f2394ab6b5a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:13.276425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:13.278083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 569/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:13 2022] GET /identity/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:13.284082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:13.287353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:13.392784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a70a-700c-4c7b-9ddb-6d9acef45ff7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:13.427569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:13.428171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:13.428448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:13.428715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:13.509290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:13.510846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:13.512684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:13.613347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 546/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:13.619679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:13.633871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:13.729279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d453d-41bd-44ad-9cbc-ecd87d153cf3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:13.770750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:13.771489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:13.771909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:13.772242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:13.773665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:13.775115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd5df2-1235-4e64-800f-b055d7b32faa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:13.776808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 570/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:13 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:13.783321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:13.888555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfc1f-f86b-4e25-b5bb-95bf9990d862 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:13.890156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:13.924716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:13.925313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:13.925603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:13.925883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:13.926661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=fca29081c46b4f309d8fd856d9bba519, user.id=cea1ee95447e4ba2b4a12f2394ab6b5a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:13.926974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:13.928419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9691f0-b7b3-4195-921d-74c116c39eac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:13.934694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 547/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:13 2022] GET /identity/v3/role_assignments?user.id=cea1ee95447e4ba2b4a12f2394ab6b5a&scope.project.id=fca29081c46b4f309d8fd856d9bba519&role.id=0faca02de5824306b70c798c90d5f033 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:15.898686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b5b9-b9a2-4e44-8f00-d3aecfb57801 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:15.899599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b5b9-b9a2-4e44-8f00-d3aecfb57801 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:15.899599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3b5b9-b9a2-4e44-8f00-d3aecfb57801 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:15.901842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 571/1118] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:15.907288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:15.907771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:15.907871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:15.928331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:15.929522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:15.998920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 548/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:16.021297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3b5b9-b9a2-4e44-8f00-d3aecfb57801 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:16.116068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:16.139882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3b5b9-b9a2-4e44-8f00-d3aecfb57801 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:16.180921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:16.181883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:16.182330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:16.182776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] PATH_INFO: `/v3/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:16.185179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] RBAC: Authorizing `identity:get_role(role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:16.187401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:16.189407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 572/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:16 2022] GET /identity/v3/roles/0faca02de5824306b70c798c90d5f033 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:16.196382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:16.307144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:16.316510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bb01e-569c-480c-936d-de6a084ff529 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:16.351448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:16.352089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:16.352365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:16.352637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] PATH_INFO: `/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:16.353966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] RBAC: Authorizing `identity:get_user(user_id=cea1ee95447e4ba2b4a12f2394ab6b5a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:16.355413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:16.356998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 549/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:16 2022] GET /identity/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:16.363092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:16.472757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d017d38-f77c-4ce7-b906-b0e8c13a69ee None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:16.512107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:16.512743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:16.513018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:16.513286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:16.514573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:16.516020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e708440-59a2-4160-8649-e80e425ba833 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:16.517783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 573/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:16 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:16.526980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:16.631365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d07e09-fd3c-42c5-b248-5e3b658124ec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:16.667090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:16.667745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:16.668022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:16.668302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519/users/cea1ee95447e4ba2b4a12f2394ab6b5a/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:16.670682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] RBAC: Authorizing `identity:create_grant(project_id=fca29081c46b4f309d8fd856d9bba519, user_id=cea1ee95447e4ba2b4a12f2394ab6b5a, role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:16.672117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb173316-9298-4be8-9865-b4118c8be750 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:16.688709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 550/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:31:16 2022] PUT /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519/users/cea1ee95447e4ba2b4a12f2394ab6b5a/roles/0faca02de5824306b70c798c90d5f033 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:31:18.644849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c258-328b-492a-bc44-f2c528ef47d6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:18.645436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c258-328b-492a-bc44-f2c528ef47d6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:18.645436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c258-328b-492a-bc44-f2c528ef47d6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:18.647662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 574/1124] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:18.656388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:18.656810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:18.656986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:18.682874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:18.684062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:18.702100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:18.753652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 551/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:18.783166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c258-328b-492a-bc44-f2c528ef47d6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:18.800202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:18.893572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c258-328b-492a-bc44-f2c528ef47d6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:18.930309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:18.930933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:18.931219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:18.931481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] PATH_INFO: `/v3/roles/0faca02de5824306b70c798c90d5f033` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:18.933037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] RBAC: Authorizing `identity:get_role(role_id=0faca02de5824306b70c798c90d5f033)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:18.934456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:18.935797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 575/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:18 2022] GET /identity/v3/roles/0faca02de5824306b70c798c90d5f033 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:18.944795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:19.044394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd0e31-2136-47bf-bd0c-26854d3ae61b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:19.079432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:19.080064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:19.080337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:19.080639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] PATH_INFO: `/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:19.081937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] RBAC: Authorizing `identity:get_user(user_id=cea1ee95447e4ba2b4a12f2394ab6b5a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:19.083374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:19.085015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 552/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:18 2022] GET /identity/v3/users/cea1ee95447e4ba2b4a12f2394ab6b5a => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:19.094505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:19.180213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:19.200362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774055e-09e5-4aeb-8e60-897152e7e389 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:19.235595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:19.236201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:19.236479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:19.236775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:19.307128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:19.309531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:19.402654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:19.424307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 576/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:19.434191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:19.461452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:19.537141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757814d-f599-47ed-b72f-8446c22a04be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:19.571272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:19.571904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:19.572182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:19.572449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] PATH_INFO: `/v3/projects/fca29081c46b4f309d8fd856d9bba519` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:19.573742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] RBAC: Authorizing `identity:get_project(project_id=fca29081c46b4f309d8fd856d9bba519)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:19.575181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c03e1-3bfc-4229-aa2e-dad45761a145 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:19.576831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 553/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:19 2022] GET /identity/v3/projects/fca29081c46b4f309d8fd856d9bba519 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:19.583599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:19.693952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22a4cd-bc98-4c11-a597-b972868e0d40 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:19.733836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:19.734373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:19.734648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:19.734916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:19.735707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] RBAC: Adding query filter params (role.id=0faca02de5824306b70c798c90d5f033, scope.project.id=fca29081c46b4f309d8fd856d9bba519, user.id=cea1ee95447e4ba2b4a12f2394ab6b5a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:19.736009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:19.737440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcdaf0-85b5-4324-9b06-046e2602187e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:19.743864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 577/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:19 2022] GET /identity/v3/role_assignments?user.id=cea1ee95447e4ba2b4a12f2394ab6b5a&scope.project.id=fca29081c46b4f309d8fd856d9bba519&role.id=0faca02de5824306b70c798c90d5f033 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:21.735608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521cac3-a4e6-4e59-90da-5a2c5a406155 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:21.736209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521cac3-a4e6-4e59-90da-5a2c5a406155 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:21.736209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521cac3-a4e6-4e59-90da-5a2c5a406155 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:21.737587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 554/1131] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:21.744068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:21.744760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:21.744936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:21.772582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:21.773821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:21.817470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:21.848630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 578/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:21.871542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521cac3-a4e6-4e59-90da-5a2c5a406155 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:21.969758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:21.981468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521cac3-a4e6-4e59-90da-5a2c5a406155 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:22.020509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:22.021161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:22.021441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:22.021746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:22.022939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:22.024387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce28f6b-22cc-47c4-b19f-5012680dae43 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:22.026027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 555/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:22.032183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:22.135953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa5abe-c9ea-4afd-beb0-0a554c79d164 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:22.172115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:22.172719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:22.172994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:22.173265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:22.173979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:22.175386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed071f-f98d-4be1-a6bd-a12234e6440a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:22.210764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 579/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:31:22 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:24.237271 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71732b-767a-4a1f-a893-38dec0f151e8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:24.238223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71732b-767a-4a1f-a893-38dec0f151e8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:24.238223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71732b-767a-4a1f-a893-38dec0f151e8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:24.240359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 556/1135] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:24.249183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:24.249641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:24.249778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:24.270119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:24.271250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:24.295317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:24.341482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 580/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:24.367295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71732b-767a-4a1f-a893-38dec0f151e8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:24.488445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71732b-767a-4a1f-a893-38dec0f151e8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:24.523724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:24.524286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:24.524565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:24.524837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:24.526138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:24.527589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9515a8-4949-4128-9005-e1b09f6805f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:24.529294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 557/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:31:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:24.535840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:24.644934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b181a39-8d6d-42fa-b6ec-79b762da23a5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:24.678853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:24.679474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:24.679753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:24.680025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:24.680706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:24.682180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456d94-4bcb-4ccb-8223-26aa56abf280 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:24.713627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 581/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:31:24 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:26.763038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06d9d-ff58-4a66-b586-fd33ee7cb0af None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:26.763593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06d9d-ff58-4a66-b586-fd33ee7cb0af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:26.763593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06d9d-ff58-4a66-b586-fd33ee7cb0af None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:26.765693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 558/1139] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:26.774501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:26.774907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:26.775111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:26.797493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:26.798724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:26.808038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:26.873678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 582/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:26.897608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f06d9d-ff58-4a66-b586-fd33ee7cb0af None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:26.938530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:27.018725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f06d9d-ff58-4a66-b586-fd33ee7cb0af None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:27.054371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:27.055001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:27.055277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:27.055543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:27.066272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:27.071143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:27.072251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:27.073177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 559/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:27.078609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:27.191379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4535a-7e87-41d0-b7e8-f30297dcc50c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:27.227418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:27.228026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:27.228306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:27.228579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:27.229288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:27.229597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:27.230997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:27.239078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 583/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:27.245063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:27.334420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:27.349922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef499e-bdf1-42c6-8c0a-36f0ea32c540 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:27.391207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:27.391838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:27.392116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:27.392384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] PATH_INFO: `/v3/users/1ff93dd50e954e81aa8377bb36c36143` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:27.406105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff93dd50e954e81aa8377bb36c36143)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:27.407538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:27.409194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 560/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:27 2022] GET /identity/v3/users/1ff93dd50e954e81aa8377bb36c36143 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:27.418399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:27.432620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:27.531321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7053-543b-4e99-956e-df968ffdc546 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:27.567464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:27.568067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:27.568337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:27.568605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:27.638421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:27.639788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:27.647293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:27.756199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 584/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:27.768813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:27.877008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004e4bd-a1b5-46e6-af58-779afbca6c17 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:27.912498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:27.913057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:27.913337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:27.913597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] PATH_INFO: `/v3/projects/cff44bcee2ea400fa37b48b405972b67` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:27.914887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] RBAC: Authorizing `identity:get_project(project_id=cff44bcee2ea400fa37b48b405972b67)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:27.916335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347010-81eb-4097-8dd6-e27044111205 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:27.918006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 561/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:27 2022] GET /identity/v3/projects/cff44bcee2ea400fa37b48b405972b67 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:27.924746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:28.042861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0724c-c4a5-42ff-a027-f03eeafc56ef None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:28.050091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:28.084736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:28.085622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:28.086011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:28.086385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:28.087604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=cff44bcee2ea400fa37b48b405972b67, user.id=1ff93dd50e954e81aa8377bb36c36143) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:28.088012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:28.090255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421deff-2376-487b-b7fe-29d4dabd9a71 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:28.098551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 585/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:27 2022] GET /identity/v3/role_assignments?user.id=1ff93dd50e954e81aa8377bb36c36143&scope.project.id=cff44bcee2ea400fa37b48b405972b67&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:30.035797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db204-a3e9-41b9-827f-91e237b9e2c6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.036541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db204-a3e9-41b9-827f-91e237b9e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.036650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db204-a3e9-41b9-827f-91e237b9e2c6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.038965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 562/1147] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:30.044473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.044932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.045026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.063911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:30.065102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:30.138055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 586/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:30.161223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90db204-a3e9-41b9-827f-91e237b9e2c6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:30.185251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:30.280919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90db204-a3e9-41b9-827f-91e237b9e2c6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:30.316492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:30.317115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.317390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.317685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.328415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:30.333299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:30.334442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:30.335447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:30.337729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 563/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:30.340738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:30.448845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:30.453393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a06c13-69ea-4d7d-8e37-03c739a4a962 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:30.494422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:30.495051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.495324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.495588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.496417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:30.496718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:30.498120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:30.506399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 587/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:30.514525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:30.620309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:30.660007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cdd85-b868-4ffa-89a6-c27383426a89 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:30.694062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:30.694682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.694971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.695238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] PATH_INFO: `/v3/users/1ff93dd50e954e81aa8377bb36c36143` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.696428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff93dd50e954e81aa8377bb36c36143)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:30.697894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:30.699481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 564/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:30 2022] GET /identity/v3/users/1ff93dd50e954e81aa8377bb36c36143 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:30.705541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:30.809481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930f33-b4b5-41c4-8f5f-0fdad88512bc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:30.848029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:30.848663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:30.848909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:30.849177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] PATH_INFO: `/v3/projects/cff44bcee2ea400fa37b48b405972b67` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:30.850421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] RBAC: Authorizing `identity:get_project(project_id=cff44bcee2ea400fa37b48b405972b67)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:30.851886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce8ee1-aa69-478c-a87c-7878abcb540b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:30.853610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 588/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:30 2022] GET /identity/v3/projects/cff44bcee2ea400fa37b48b405972b67 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:30.860644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:30.969842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9ebc-5afa-4c8d-84d6-55b5df682182 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:31.005744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:31.006436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:31.006741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:31.007012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] PATH_INFO: `/v3/projects/cff44bcee2ea400fa37b48b405972b67/users/1ff93dd50e954e81aa8377bb36c36143/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:31.009417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff44bcee2ea400fa37b48b405972b67, user_id=1ff93dd50e954e81aa8377bb36c36143, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:31.010920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6004bf-9557-41ae-a508-6a382f4d33d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:31.025075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:31.031251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 565/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:31:30 2022] PUT /identity/v3/projects/cff44bcee2ea400fa37b48b405972b67/users/1ff93dd50e954e81aa8377bb36c36143/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:31:32.991516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1529-6665-4199-87bc-dcb5d5e30584 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:32.992466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1529-6665-4199-87bc-dcb5d5e30584 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:32.992466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1529-6665-4199-87bc-dcb5d5e30584 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:32.994642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 589/1154] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:33.000346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:33.000769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:33.000873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:33.022698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:33.023910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:33.092663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 566/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:33.123248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1529-6665-4199-87bc-dcb5d5e30584 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:33.155242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:33.239642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1529-6665-4199-87bc-dcb5d5e30584 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:33.281275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:33.281851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:33.282125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:33.282390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:33.293827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:33.298753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:33.300107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:33.300971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 590/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:33.306414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:33.409382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32b19-fe4a-4d91-9431-50111fb589ad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:33.441972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:33.448799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:33.449411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:33.449720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:33.449991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:33.450714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:33.451012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:33.452386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:33.461148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 567/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:33.467329 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:33.562256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:33.571942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36ff88-dd11-4758-a5c9-238f84ef7c55 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:33.610308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:33.610927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:33.611208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:33.611467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] PATH_INFO: `/v3/users/1ff93dd50e954e81aa8377bb36c36143` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:33.612693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff93dd50e954e81aa8377bb36c36143)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:33.614146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:33.615734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 591/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:33 2022] GET /identity/v3/users/1ff93dd50e954e81aa8377bb36c36143 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:33.621821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:33.735696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e59c9-b44e-4037-a869-8970eeece390 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:33.771691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:33.772333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:33.772611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:33.772890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:33.835044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:33.847974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:33.849447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:33.953264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 568/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:33.960301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:34.079700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e55d3-a99f-437d-80b5-dae8d3c8ee29 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:34.114326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:34.114883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:34.115158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:34.115430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] PATH_INFO: `/v3/projects/cff44bcee2ea400fa37b48b405972b67` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:34.116693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] RBAC: Authorizing `identity:get_project(project_id=cff44bcee2ea400fa37b48b405972b67)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:34.118155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508eebc7-67da-4b3b-9fec-42ceb03d56c6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:34.119843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 592/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:33 2022] GET /identity/v3/projects/cff44bcee2ea400fa37b48b405972b67 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:34.126242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:34.219390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:34.229445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8787-4781-44e2-b2e3-98d6ed86f387 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:34.265421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:34.266055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:34.266328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:34.266594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:34.267356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=cff44bcee2ea400fa37b48b405972b67, user.id=1ff93dd50e954e81aa8377bb36c36143) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:34.267659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:34.269107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da807cb2-238f-42c7-ad78-e07b36b46e82 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:34.274981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 569/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:34 2022] GET /identity/v3/role_assignments?user.id=1ff93dd50e954e81aa8377bb36c36143&scope.project.id=cff44bcee2ea400fa37b48b405972b67&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:36.266501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34435-2846-47df-9ea6-b76420e23cab None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:36.267019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34435-2846-47df-9ea6-b76420e23cab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:36.267019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34435-2846-47df-9ea6-b76420e23cab None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:36.269031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 593/1162] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:36.277890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:36.278287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:36.278510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:36.301378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:36.302594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:36.372114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 570/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:36.395009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e34435-2846-47df-9ea6-b76420e23cab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:36.439828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:36.526611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e34435-2846-47df-9ea6-b76420e23cab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:36.561262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:36.561906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:36.562178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:36.562442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:36.630590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:36.632014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:36.657942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:36.746579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 594/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:36.756554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:36.848743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:36.863360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235f0a3-4037-446f-952b-9df24ddee6c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:36.896780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:36.897421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:36.897738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:36.898015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:36.904652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:36.906145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-36eb00b7-30ae-4d4e-967d-c599cfac1e7a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:36.917693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:36.918825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:36.920110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 571/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:31:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:36.927587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:37.039794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdeb8f-1add-47aa-b33d-8a33804d13ea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:37.071130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:37.081527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:37.082185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:37.082460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:37.082758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:37.083454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:37.083753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:37.085122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcacc8-7a15-4b14-b2ec-fe517b0b7b06 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:37.092298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 595/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Mon Aug 1 22:31:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:39.069393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6dcff-dfaf-4c20-8722-097fb462db69 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:39.069918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6dcff-dfaf-4c20-8722-097fb462db69 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:39.069918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6dcff-dfaf-4c20-8722-097fb462db69 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:39.071337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 572/1167] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:39.077613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbac82-ece1-4542-bff5-879aa7010176 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:39.078259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbac82-ece1-4542-bff5-879aa7010176 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:39.078531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbac82-ece1-4542-bff5-879aa7010176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:39.103334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7dcbac82-ece1-4542-bff5-879aa7010176 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:39.104523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7dcbac82-ece1-4542-bff5-879aa7010176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:39.176082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 596/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:39.205412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6dcff-dfaf-4c20-8722-097fb462db69 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:39.338940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6dcff-dfaf-4c20-8722-097fb462db69 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:39.373388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:39.374021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:39.374301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:39.374564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:39.375260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:39.376625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a694497-00cb-49dd-8e0d-ec65cc66c85b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:39.394910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 573/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Mon Aug 1 22:31:39 2022] POST /identity/v3/domains => generated 267 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:41.425059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aa994-dbe1-4ea4-b6d3-0f2bfdfa4bb8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:41.425666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aa994-dbe1-4ea4-b6d3-0f2bfdfa4bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:41.425666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aa994-dbe1-4ea4-b6d3-0f2bfdfa4bb8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:41.427077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 597/1170] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:41.433533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:41.434378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:41.434524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:41.451956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:41.458097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:41.459840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:41.539055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 574/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:41.562600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aa994-dbe1-4ea4-b6d3-0f2bfdfa4bb8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:41.683736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aa994-dbe1-4ea4-b6d3-0f2bfdfa4bb8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:41.705367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:41.720669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:41.721319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:41.721678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:41.721948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] PATH_INFO: `/v3/domains/960baccdee4f4daa820eebc6e8e4990a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:41.733600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=960baccdee4f4daa820eebc6e8e4990a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:41.735108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041120f-61cd-4101-abca-5082db04ec0c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:41.736786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 598/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Mon Aug 1 22:31:41 2022] GET /identity/v3/domains/960baccdee4f4daa820eebc6e8e4990a => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:41.742923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:41.852744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e355c41-ea86-4189-b39c-4ac8fbaaac1b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:41.891647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:41.892257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:41.892536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:41.892786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:41.893475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:41.895072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94decc1a-7ee4-4ab9-baef-2866f55119ad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:41.901067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:41.926235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 575/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:31:41 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:43.982859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf95d-a03e-4678-97a8-ec46b3d5cd70 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:43.983760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf95d-a03e-4678-97a8-ec46b3d5cd70 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:43.983760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf95d-a03e-4678-97a8-ec46b3d5cd70 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:43.986022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 599/1174] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:43.992700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:43.993125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:43.993241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:44.017253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:44.018491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:44.098364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 576/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:44.123047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf95d-a03e-4678-97a8-ec46b3d5cd70 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:44.232839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf95d-a03e-4678-97a8-ec46b3d5cd70 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:44.266941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:44.267498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:44.267768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:44.268047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] PATH_INFO: `/v3/domains/960baccdee4f4daa820eebc6e8e4990a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:44.269299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=960baccdee4f4daa820eebc6e8e4990a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:44.270859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df132dda-08c3-42cc-88ff-1d9556f89fa9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:44.272436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 600/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Mon Aug 1 22:31:44 2022] GET /identity/v3/domains/960baccdee4f4daa820eebc6e8e4990a => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:44.279190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:44.393428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a368ec-dcb8-4363-a345-4127567855b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:44.410779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:44.430646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:44.431257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:44.431538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:44.431807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:44.432483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:44.433949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e25e95-7757-44de-ae80-bb4db7459d3e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:44.456456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 577/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:31:44 2022] POST /identity/v3/users => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:31:46.491247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484fd54-cfc7-44f5-8a41-e30540461eea None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:46.492181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484fd54-cfc7-44f5-8a41-e30540461eea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:46.492181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484fd54-cfc7-44f5-8a41-e30540461eea None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:46.494390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 601/1178] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:46.503207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:46.503654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:46.503776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:46.529308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:46.530542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:46.600264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 578/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:46.624058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484fd54-cfc7-44f5-8a41-e30540461eea None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:46.683523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:46.747190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484fd54-cfc7-44f5-8a41-e30540461eea None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:46.782771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:46.783401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:46.783677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:46.783946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:46.795216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:46.800121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:46.801261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:46.802276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 602/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:46.807728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:46.909056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4adf7e-f3de-4b82-b694-470ad713a9dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:46.911466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:46.944043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:46.944655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:46.944934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:46.945207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:46.945959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:46.946270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:46.947664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:46.955708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 579/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:46.961788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:47.069435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef77c8-5ec2-47d5-b18f-6d6ebb09d4b7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:47.078309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:47.108703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:47.109335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:47.109613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:47.109917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] PATH_INFO: `/v3/users/d0664aa01f504751976590f38aef8ebb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:47.125140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] RBAC: Authorizing `identity:get_user(user_id=d0664aa01f504751976590f38aef8ebb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:47.126621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:47.128280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 603/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:46 2022] GET /identity/v3/users/d0664aa01f504751976590f38aef8ebb => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:47.134554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:47.239622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3d796-36d9-4a0d-a6ce-b88139e29efb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:47.283712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:47.284253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:47.284533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:47.284807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:47.320404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:47.365157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:47.366647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:47.473223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 580/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:47.479638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:47.587855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f2387-015d-4489-8683-1aa19d325a64 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:47.623387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:47.624013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:47.624291 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:47.624558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] PATH_INFO: `/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:47.625902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] RBAC: Authorizing `identity:get_project(project_id=b78ba31c7b5545e0bb2c8f72d3513896)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:47.627349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de78b81-0ed3-4fbf-a8ac-18b445b93769 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:47.629000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 604/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:47 2022] GET /identity/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:47.636008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:47.709170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:47.745436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a4029-f5a4-4950-8429-52401fc662fb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:47.780278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:47.780882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:47.781160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:47.781430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:47.782252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=b78ba31c7b5545e0bb2c8f72d3513896, user.id=d0664aa01f504751976590f38aef8ebb) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:47.782558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:47.783987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed49ee-084f-4069-a6c4-736601681fa0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:47.789597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 581/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:47 2022] GET /identity/v3/role_assignments?user.id=d0664aa01f504751976590f38aef8ebb&scope.project.id=b78ba31c7b5545e0bb2c8f72d3513896&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:49.734500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981cf5d-7f12-4534-8bf2-c5aa0e67e535 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:49.735097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981cf5d-7f12-4534-8bf2-c5aa0e67e535 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:49.735097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981cf5d-7f12-4534-8bf2-c5aa0e67e535 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:49.736401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 605/1186] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:49.742823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:49.743516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:49.743685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:49.772991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:49.774819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:49.848546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 582/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:49.882979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981cf5d-7f12-4534-8bf2-c5aa0e67e535 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:49.922661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:49.995500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981cf5d-7f12-4534-8bf2-c5aa0e67e535 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:50.040284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:50.040934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:50.041208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:50.041476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:50.052959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:50.062352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:50.063357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:31:50.064421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 606/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:50.074369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:50.143499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:50.187551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e871-86c6-4197-898a-f6861561fcfc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:50.225586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:50.226223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:50.226504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:50.226777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:50.227495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:50.227795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:50.229182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:50.237319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 583/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:50.243617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:50.341601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:50.349232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dffeaa-a7dd-4ded-99d3-1188fb8d91dc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:50.385156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:50.385812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:50.386095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:50.386365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] PATH_INFO: `/v3/users/d0664aa01f504751976590f38aef8ebb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:50.387582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] RBAC: Authorizing `identity:get_user(user_id=d0664aa01f504751976590f38aef8ebb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:50.389016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:50.390669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 607/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:50 2022] GET /identity/v3/users/d0664aa01f504751976590f38aef8ebb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:50.400159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:50.520021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2898-99bc-4343-b8af-c245877e6ea1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:50.545904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:50.560074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:50.560744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:50.561022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:50.561292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] PATH_INFO: `/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:50.562763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] RBAC: Authorizing `identity:get_project(project_id=b78ba31c7b5545e0bb2c8f72d3513896)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:50.564226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4d926-8e87-4fde-b45b-bef7584beb04 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:50.566066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 584/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:50 2022] GET /identity/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896 => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:50.572532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:50.687197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21587e-898f-40dc-af61-70321b0616d6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:50.723865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:50.724523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:50.724792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:50.725059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] PATH_INFO: `/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896/users/d0664aa01f504751976590f38aef8ebb/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:50.727252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] RBAC: Authorizing `identity:create_grant(project_id=b78ba31c7b5545e0bb2c8f72d3513896, user_id=d0664aa01f504751976590f38aef8ebb, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:50.728690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3caa81-e822-4c2d-b47d-193ba647cf53 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:50.744372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 608/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:31:50 2022] PUT /identity/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896/users/d0664aa01f504751976590f38aef8ebb/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:31:52.692135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466ed4-5a5e-41b6-9601-7a8ce09ad1e7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:52.692725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466ed4-5a5e-41b6-9601-7a8ce09ad1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:52.692725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8466ed4-5a5e-41b6-9601-7a8ce09ad1e7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:52.694650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 585/1193] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:31:52.701068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:52.701529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:52.701664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:52.728249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:31:52.729456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:31:52.750870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:52.813145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 609/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:31:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:31:52.836743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8466ed4-5a5e-41b6-9601-7a8ce09ad1e7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:52.950875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8466ed4-5a5e-41b6-9601-7a8ce09ad1e7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:52.996172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:52.996813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:52.997091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:52.997361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:53.017583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:31:53.023788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:31:53.024935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:31:53.026086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 586/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:31:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:31:53.031938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:53.154612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279e902-c05f-4b64-b352-24d6753582fa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:53.188843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:53.189451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:53.189760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:53.190026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:53.190735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:53.191035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:53.192406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:53.200423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 610/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:31:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:53.209294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:53.262965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:53.318351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0652f-a426-4b0d-b2b0-9aa503d39454 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:53.363815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:53.364489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:53.364765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:53.365039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] PATH_INFO: `/v3/users/d0664aa01f504751976590f38aef8ebb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:53.366300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] RBAC: Authorizing `identity:get_user(user_id=d0664aa01f504751976590f38aef8ebb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:53.367747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:53.369351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 587/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Mon Aug 1 22:31:53 2022] GET /identity/v3/users/d0664aa01f504751976590f38aef8ebb => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:53.375631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:53.393789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:53.490732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f4f2-7001-4e32-b3ca-48aebf97ed4d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:53.532219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:53.532823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:53.533093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:53.533366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:53.600396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:53.601908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:53.616199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:53.716579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 611/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:31:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:31:53.723435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:53.833371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2849-4400-445e-bc35-c867fe076855 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:53.852954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:53.875086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:53.875728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:53.876005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:53.876272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] PATH_INFO: `/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:53.877475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b78ba31c7b5545e0bb2c8f72d3513896)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:53.878962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353fd0-b2b9-4c0c-ae05-b11b010844d4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:53.880620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 588/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Mon Aug 1 22:31:53 2022] GET /identity/v3/projects/b78ba31c7b5545e0bb2c8f72d3513896 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:31:53.886985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:31:53.988183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfde1f-619c-4bda-8602-0d8a5662d881 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:31:54.002038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:31:54.024757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:31:54.025372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:31:54.025646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:31:54.025929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:31:54.026718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=b78ba31c7b5545e0bb2c8f72d3513896, user.id=d0664aa01f504751976590f38aef8ebb) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:31:54.027022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:31:54.028445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d643bfa-4428-4702-bcfb-75eb0d5757d3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:31:54.034215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 612/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:31:53 2022] GET /identity/v3/role_assignments?user.id=d0664aa01f504751976590f38aef8ebb&scope.project.id=b78ba31c7b5545e0bb2c8f72d3513896&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:03.870047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:04.031674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=11 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:48.910429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9941-23f1-4797-9be3-af37058a3469 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:48.911063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9941-23f1-4797-9be3-af37058a3469 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:48.911063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a9941-23f1-4797-9be3-af37058a3469 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:48.912585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 589/1201] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:32:48.919777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:48.920692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:48.920692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:48.942183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:32:48.944513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:32:49.032226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 613/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:32:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:32:49.059786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a9941-23f1-4797-9be3-af37058a3469 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:49.194615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a9941-23f1-4797-9be3-af37058a3469 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:49.245170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:49.246082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:49.246451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:49.246815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:49.259896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:49.262338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:32:49.284176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:49.285309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:49.286314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:32:49.286314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:32:49.286314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:49.286314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:32:49.286314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 590/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:49.292334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:49.425093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3fc9-be54-48d3-97d1-86e023517a2f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:49.458302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:49.466162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:49.466162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:49.466162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:49.466162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:49.466623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:49.466623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:49.467935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aec5c-12b8-4523-8670-420da7404464 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:49.480743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 614/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:49.486352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:49.548403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:49.605834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904ccc-e79b-4c6e-b4c4-b0e12f373355 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:49.654761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:49.655387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:49.655669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:49.655962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:49.656632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:49.658111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21447f95-5fb7-408e-9dda-f9f2159f98aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:49.681368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 591/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Mon Aug 1 22:32:49 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:32:51.822193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47475c-398e-4ad8-bf9d-8faa9fa28eb9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:51.822809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47475c-398e-4ad8-bf9d-8faa9fa28eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:51.822809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47475c-398e-4ad8-bf9d-8faa9fa28eb9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:51.824265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 615/1206] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:32:51.830183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:51.830628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:51.830715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:51.850997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:32:51.852344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:32:51.927945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 592/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:32:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:32:51.953345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47475c-398e-4ad8-bf9d-8faa9fa28eb9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:52.083747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47475c-398e-4ad8-bf9d-8faa9fa28eb9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:52.121278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:52.122010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:52.122288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:52.122518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:52.134574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:32:52.140311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:52.141356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:52.142267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 616/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:32:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:52.148147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:52.191848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:52.265573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a68c75-4e2f-459e-b86c-f1f5d4a65b63 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:52.305766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:52.306407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:52.306686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:52.306959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:52.307689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:52.308026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:52.309706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:52.319002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 593/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:32:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:32:52.325001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:52.384375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:52.454691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e55fa-28a0-4072-8d0c-876fad9e6907 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:52.493382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:52.494237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:52.494521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:52.494788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:52.506606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:52.508169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:32:52.524318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:52.525278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:52.526330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:32:52.526330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:32:52.526330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:52.526330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:52.526330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 617/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:32:52.532520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:52.648746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:52.659635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638a516-5ad2-41aa-acab-340c97dde937 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:52.698399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:52.701595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:52.701595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:52.701595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:52.701595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:52.701595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:52.702256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:52.718254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 594/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:52.724454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:52.845843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b66a-b6b4-4bc8-bd75-7ab448d42cad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:52.847674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:52.886230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:52.886931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:52.887394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:52.887786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:52.896969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:52.898581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:32:52.911328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:52.912330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:52.913376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 618/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:52.918723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:53.043583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8a358-9f12-49c5-af4b-0c39e69e05ac None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:53.080286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:53.080940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:53.081220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:53.081491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:53.082325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:53.082630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:53.084539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:53.103249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 595/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:32:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:32:53.109246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:53.223088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d16c8-953a-4699-bcac-a72c87a49283 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:53.258155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:53.258818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:53.259046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:53.259313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:53.293566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:53.335746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:53.337079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:53.446550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 619/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:32:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:32:53.452410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:53.475369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:53.578381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902b047-9a3b-4de6-beff-5ebcaf7309e0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:53.617318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:53.618063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:53.618340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:53.618668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:53.632196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:53.633751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:32:53.650488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:53.652257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:53.653521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:32:53.653521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:32:53.653521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:53.653521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:32:53.653521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 596/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:53.658138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:53.735070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:53.786531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5f434-1b20-4048-a1c9-a4ae14bcaeeb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:53.826044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:53.826708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:53.826990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:53.827258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:53.828101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:53.828402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:53.830007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:53.847447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 620/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:53.854045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:53.935316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:53.972712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5321-b614-41bd-a9a0-a6a3d1fd108d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:54.012881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:54.013690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:54.014104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:54.014753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:54.025375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:54.027255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:32:54.040574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:54.042247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:32:54.043847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:32:54.045015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 597/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:54.051222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:54.177378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a7738-380a-49d6-955a-142aa8206f83 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:54.203175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:54.217288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:54.217984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:54.218290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:54.218634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:54.219451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:54.219771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:54.221369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc362-af8a-472d-b55d-36b9b3ec14ef None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:54.235227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 621/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:54.242011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:54.353727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3937-af03-478a-9654-b7fea6329081 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:54.384673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:54.394049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:54.394770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:54.395152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:54.395495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:54.396472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=30e6ac9cc1264e109a68bc20fd975623) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:54.396840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:54.398578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6924a2-b3a5-40b6-a295-d93050ba6598 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:54.409638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 598/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:32:54 2022] GET /identity/v3/role_assignments?user.id=30e6ac9cc1264e109a68bc20fd975623&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:56.443834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd045c-c9d6-4919-8a26-8d9d6cbbe348 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:56.444452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd045c-c9d6-4919-8a26-8d9d6cbbe348 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:56.444452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd045c-c9d6-4919-8a26-8d9d6cbbe348 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:56.445836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 622/1220] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:32:56.451723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:56.452275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:56.452472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:56.476542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:32:56.478022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:32:56.564742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 599/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:32:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:32:56.587523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd045c-c9d6-4919-8a26-8d9d6cbbe348 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:56.707502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd045c-c9d6-4919-8a26-8d9d6cbbe348 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:56.746669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:56.747427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:56.747797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:56.748100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:56.761447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:32:56.765991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:56.767030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:56.767953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 623/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:56.774308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:56.890746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fcf2a-bab8-4918-bf5c-04e261cf306c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:56.922014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:56.934245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:56.934960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:56.935322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:56.935706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:56.936595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:56.936904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:56.938461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:56.948240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 600/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:56.954227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.004899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:57.070745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c8d68-1d8d-4d95-a50f-b09dbb3d6698 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.111592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:57.112404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:57.112764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:57.113059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:57.124877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:57.126391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:32:57.141488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:57.142497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:57.143900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:32:57.143900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:32:57.143900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:57.143900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:57.143900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 624/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:57.148668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.261769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b600c0-d4d2-4ed1-922f-b4652a1d518b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.299283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:57.300156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:57.300332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:57.300778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:57.301671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:57.302145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:57.304010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:57.318733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 601/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:57.326232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.426928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:57.434992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ee56-fa43-40b0-9ef5-e63226d9bf84 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.470535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:57.471302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:57.471635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:57.472005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:57.481669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:57.483378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:32:57.498226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:57.499390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:57.500507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 625/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:57.505649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.555521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:57.613695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a8c6-b925-473b-8b2b-d1c9e58109d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.652565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:57.653199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:57.653477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:57.653780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:57.654585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:57.654896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:57.656336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:57.673800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 602/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:32:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:57.679715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.787723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62903ee-710e-438b-9aec-ac1b3b6588d6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.822631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:57.823194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:57.823471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:57.823743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:57.834633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:57.836125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:32:57.852988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:57.854212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:57.855173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:32:57.855173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:32:57.855173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:32:57.855173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:57.855173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 626/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:57.860705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:57.967397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce7923-f3ca-4313-9948-29b473d2591e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:57.969797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:58.011524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:58.012296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:58.012602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:58.012872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:58.013594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:58.013896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:58.015552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:58.028242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 603/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:58.034453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:58.044346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:58.146759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eed35-1ac8-4a61-b614-71dedb3d4efb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:58.181216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:58.181963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:58.182265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:58.182568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:58.189693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:58.191242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:32:58.203201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:32:58.204563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:32:58.205746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 627/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:32:58.211288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:58.328263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538a0c0-2c78-4c4e-bc18-9839527aa399 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:58.369062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:58.369833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:58.370132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:58.370403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:58.371281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:32:58.371703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:58.373195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b8350-f655-4755-8e6a-4b7844e190c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:58.386571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 604/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:32:58.392852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:32:58.473272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:32:58.515476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870df31-43f3-4add-a1fc-09a7d8217577 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:32:58.553320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:32:58.554135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:32:58.554416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:32:58.554804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/30e6ac9cc1264e109a68bc20fd975623/roles/b4cdc113ee3041739d9a48eadc3f6437` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:32:58.571119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=30e6ac9cc1264e109a68bc20fd975623, role_id=b4cdc113ee3041739d9a48eadc3f6437)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:32:58.572587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a2f7a-c1e6-4b94-ba8a-9cc93bce169c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:32:58.585724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 628/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:32:58 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/30e6ac9cc1264e109a68bc20fd975623/roles/b4cdc113ee3041739d9a48eadc3f6437 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:33:00.806409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c69408-c1a6-4414-8a16-48e0dd859ae0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:00.807044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c69408-c1a6-4414-8a16-48e0dd859ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:00.807044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c69408-c1a6-4414-8a16-48e0dd859ae0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:00.809589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 605/1233] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:00.814195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:00.814669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:00.814774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:00.833822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:00.835039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:00.906583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 629/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:00.932668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c69408-c1a6-4414-8a16-48e0dd859ae0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:00.957466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:01.049979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c69408-c1a6-4414-8a16-48e0dd859ae0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.095262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.095905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.096172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.096451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.111349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:33:01.117730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:01.118923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:01.120009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 606/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Mon Aug 1 22:33:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:01.128543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:01.180083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:01.245704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c018cb-7f28-45c5-81c7-15226a59637f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.285277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.286041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.286542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.288366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.288742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:01.289053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:01.290878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:01.300403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 630/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Mon Aug 1 22:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:01.306528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:01.398274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:01.415171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e82015-95dd-4d19-81e6-aec1eead6a2a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.449261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.449972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.450231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.450528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.460988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:01.462504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:01.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:01.480507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:01.481456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:01.481456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:01.481456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:01.481456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:01.481456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 607/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:01.486888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:01.592760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e379a-10cb-43a7-96b7-cd8441d669b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.605297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:01.632710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.633334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.633639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.633916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.634615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:01.634917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:01.636292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:01.648435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 631/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:01.655491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:01.774389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bebba-5b66-42b2-a3db-ea2b592fd209 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.812167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.812913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.813192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.813438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.816268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:01.821598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:01.823245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:33:01.834493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:01.835621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:01.836572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 608/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:01.841968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:01.950917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3df7-37e4-4da5-8cd2-fe361bef2ae4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:01.987096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:01.987758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:01.988049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:01.988413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:01.989342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:01.989801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:01.991667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:02.013819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 632/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:33:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:33:02.020341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:02.132683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da86c5-3805-4a40-96cb-e3afdb63a09e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:02.169121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:02.169772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:02.170057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:02.170350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:02.237202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:02.245180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:02.246897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:02.352043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 609/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:33:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:33:02.358222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:02.390231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:02.469918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f8b4-ee18-4fe9-a693-59773d1b9c97 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:02.508769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:02.509435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:02.509794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:02.510070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:02.521279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:02.522800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:02.538578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:02.539660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:02.540758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:02.540758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:02.540758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:02.540758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:02.540758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 633/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:02.545909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:02.647671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:02.653826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95783-14a5-4b65-95c6-492816e90a98 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:02.696197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:02.696824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:02.697109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:02.697383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:02.698111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:02.698409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:02.699819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:02.711443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 610/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:02.717527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:02.812845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:02.837319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7945e-616d-4f27-a469-2d305513a130 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:02.873980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:02.874657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:02.874867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:02.875138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:02.882206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:02.883769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:33:02.896599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:02.897773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:02.898802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:02.901023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 634/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:02.908068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:03.012757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bdf61-d53f-4b6b-8cbc-3f233de56f1e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:03.047949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:03.048603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:03.048878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:03.049157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:03.049945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:03.050251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:03.051706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546061a-73a6-430a-b564-2bd230b1c435 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:03.063956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 611/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:33:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:03.070614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:03.178398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4ac4-3ace-4c7a-b3e5-c5fc1e9d53a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:03.214895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:03.215556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:03.215792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:03.216101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:03.216870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] RBAC: Adding query filter params (role.id=b4cdc113ee3041739d9a48eadc3f6437, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=30e6ac9cc1264e109a68bc20fd975623) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:03.217176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:03.218684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751d4b-b223-403d-b1b7-933afc6d50fb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:03.226137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 635/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:33:03 2022] GET /identity/v3/role_assignments?user.id=30e6ac9cc1264e109a68bc20fd975623&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=b4cdc113ee3041739d9a48eadc3f6437 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:05.397610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f18fb-a247-4b13-93d5-1d9735b16b4e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:05.398186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f18fb-a247-4b13-93d5-1d9735b16b4e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:05.398186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f18fb-a247-4b13-93d5-1d9735b16b4e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:05.399625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 612/1247] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:05.405799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:05.406421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:05.406421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:05.416309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:05.430004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:05.431553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:05.511562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 636/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:05.536123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f18fb-a247-4b13-93d5-1d9735b16b4e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:05.555288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:05.654727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f18fb-a247-4b13-93d5-1d9735b16b4e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:05.694807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:05.695461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:05.695751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:05.696013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:05.708787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:33:05.714043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:05.715209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:05.716398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 613/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:05.721447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:05.841573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c376-4b58-428b-b226-7d13cb67aa8f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:05.874063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:05.881270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:05.882017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:05.882343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:05.882573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:05.883297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:05.883598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:05.884990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:05.894322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 637/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:05.900702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:05.986909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:06.011936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814e90-a814-4d43-9348-3ec368c58ce6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:06.050503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:06.051269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:06.051638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:06.051967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:06.063390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:06.064931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:06.080824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:06.082035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:06.082967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:06.082967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:06.082967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:06.082967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:06.082967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 614/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:06.088412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:06.190421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e87e36-4cbc-4672-9e02-63086cf9d770 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:06.223930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:06.224553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:06.224826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:06.225461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:06.226281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:06.226582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:06.227949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:06.241054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 638/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:06.246719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:06.355836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c866-ccd7-46c1-98c4-4ab6769bf49b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:06.384416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:06.393057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:06.393760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:06.394069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:06.394345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:06.402171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:06.403644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:33:06.415789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:06.417000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:06.418374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 615/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:06.423862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:06.490162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:06.538807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce1ad-b21e-4148-9e39-e763232ff839 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:06.575905 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:06.576564 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:06.576840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:06.577114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:06.577990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:06.578292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:06.579772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:06.600372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 639/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:33:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:06.606452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:06.729609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e32c3e-f47f-4a11-b156-4aa8463c040c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:06.776194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:06.776907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:06.777188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:06.777468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:06.843390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:06.858367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:06.860261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:06.967750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 616/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:33:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:33:06.973779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:07.087559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b17f5c-eecf-43ad-bb15-56a075d189e2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:07.105024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:07.133123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:07.133819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:07.134098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:07.134374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:07.148090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:07.150206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:07.165278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:07.166535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:07.167659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:07.167659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:07.167659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:07.167659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:07.167659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 640/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 22:33:07.172371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:07.268210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:07.281503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236e247-7531-41ea-a534-83dc489c0843 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:07.320710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:07.321350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:07.321702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:07.321936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:07.322645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:07.322936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:07.324361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:07.338572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 617/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:07.344522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:07.459507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26e444-e9ea-4301-8f44-1f5e779a3267 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:07.497568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:07.498245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:07.498520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:07.498791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:07.506021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:07.507506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:33:07.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:07.521201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:07.522258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 641/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:07.527367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:07.646273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a038fa-d35a-429f-8b7b-48d5bd24f230 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:07.686901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:07.687822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:07.688189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:07.688522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:07.689918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:07.690419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:07.692585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fbcf9-06c1-4faf-bf84-0155ef2f1600 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:07.708058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 618/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:07.714477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:07.747202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:07.843253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b8716-4199-48d1-8511-954fcc1a67f8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:07.880005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:07.880674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:07.880956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:07.881351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:07.882319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=30e6ac9cc1264e109a68bc20fd975623) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:07.882754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:07.884712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecc082-d721-4e18-bc3c-f6b90464c158 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:07.891165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 642/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:33:07 2022] GET /identity/v3/role_assignments?user.id=30e6ac9cc1264e109a68bc20fd975623&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:09.882458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1744-7679-48d1-856e-6cb7d13ec523 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:09.883051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1744-7679-48d1-856e-6cb7d13ec523 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:09.883051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1744-7679-48d1-856e-6cb7d13ec523 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:09.884509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 619/1261] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:09.889918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187210-cd0d-4b16-b190-030034577024 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:09.890345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187210-cd0d-4b16-b190-030034577024 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:09.890458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187210-cd0d-4b16-b190-030034577024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:09.909008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2e187210-cd0d-4b16-b190-030034577024 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:09.910265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2e187210-cd0d-4b16-b190-030034577024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:09.984020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 643/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:10.006408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe1744-7679-48d1-856e-6cb7d13ec523 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:10.018025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:10.117880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe1744-7679-48d1-856e-6cb7d13ec523 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:10.150047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:10.150619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:10.150893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:10.151160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:10.161477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:33:10.166466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:10.167615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:10.169136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 620/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:10.177175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187210-cd0d-4b16-b190-030034577024 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:10.202908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:10.292660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187210-cd0d-4b16-b190-030034577024 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:10.328034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:10.328647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:10.328925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:10.329197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:10.329963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:10.330269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:10.331645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:10.339922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 644/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:10.348763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:10.421784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:10.459026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a85fd6-5cdd-4caa-9247-934eecabf3e9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:10.494423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:10.495074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:10.495352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:10.495626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:10.506316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:10.507871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:10.522910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:10.523925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:10.532600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:10.532600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:10.532600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:10.532600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:10.532600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 621/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:10.533219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:10.625216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:10.644906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d4e0-d70d-42f8-bcd9-c0d7245722dd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:10.680595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:10.681145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:10.681430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:10.681730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:10.682424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:10.682728 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:10.684083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:10.695626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 645/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:10.701676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:10.821607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74e8fa-a68b-47be-aae7-27ca9c480c01 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:10.842640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:10.856984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:10.857618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:10.857917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:10.858187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:10.865130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:10.866630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:33:10.878232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:10.879309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:10.880434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 622/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:10.886077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.000125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77550e-7bc0-4860-b759-2eff5cdf7462 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.037324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.037981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.038284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.038551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.039348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:11.039649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.041090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.049831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:11.060602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 646/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:33:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:11.066536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.172739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad65-4298-4c23-95dc-03a813fa96bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.214843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.215486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.215764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.216033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.227443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.228890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:11.243367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:11.244403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:11.245510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:11.245510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:11.245510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:11.245510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:11.245510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 623/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:11.254753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.365587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26609-974b-4be0-a6ea-5e5a1fa7d1b5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.400863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.401474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.401784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.402057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.402753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:11.403049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.404422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.415829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 647/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:11.422512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.467374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:11.543421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcd078-66ed-4906-827b-b4bf6d788286 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.580470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.581094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.581370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.581659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.589180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.590675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:33:11.603337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:11.604471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:11.605447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 624/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:11.610724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.679432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:11.721201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd640d2f-6d08-42a3-a7bc-7fdd21d00c52 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.761963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.762868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.763265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.763545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.764458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:11.764864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.766680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45155-a192-4c5d-b684-5d51bbdf37b0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.780787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 648/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:11.787194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:11.887039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:11.897104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbec936-7020-4019-8048-214b00ced3fb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:11.931547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:11.932206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:11.932480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:11.932746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] PATH_INFO: `/v3/projects/668cc085aa8f430e93314a3a57211a59/users/30e6ac9cc1264e109a68bc20fd975623/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:11.934782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] RBAC: Authorizing `identity:create_grant(project_id=668cc085aa8f430e93314a3a57211a59, user_id=30e6ac9cc1264e109a68bc20fd975623, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:11.936222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6327e-ad8b-45bf-91e4-4a75d6c02abc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:11.952495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 625/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Mon Aug 1 22:33:11 2022] PUT /identity/v3/projects/668cc085aa8f430e93314a3a57211a59/users/30e6ac9cc1264e109a68bc20fd975623/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:33:13.923725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412a6a-bbf0-4c02-8334-e3aba5e5c45f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:13.924294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412a6a-bbf0-4c02-8334-e3aba5e5c45f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:13.924617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412a6a-bbf0-4c02-8334-e3aba5e5c45f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:13.926279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 649/1274] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:13.934999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:13.935724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:13.935870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:13.959210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:13.960444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:14.032447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 626/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:14.056201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412a6a-bbf0-4c02-8334-e3aba5e5c45f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.168968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412a6a-bbf0-4c02-8334-e3aba5e5c45f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:14.185765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:14.205167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:14.205823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:14.206103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:14.206372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:14.217245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 22:33:14.222206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:14.223419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:14.224606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 650/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:14.230102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.316507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:14.336000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35a65c-7810-4b90-826e-fbb8e310d04c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:14.370798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:14.371409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:14.371763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:14.372059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:14.372781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:14.373082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:14.374481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:14.382563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 627/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:14.388940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.500907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23eb96-e891-459a-89b9-3ec4b64c050b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:14.536265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:14.536893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:14.537168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:14.537445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:14.547477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:14.548922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:14.564437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:14.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:14.567613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:14.567613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:14.567613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:14.567613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:14.567613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 651/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:14.573104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.677286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d776-fe29-4674-b34b-86af8759af83 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:14.712516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:14.713131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:14.713416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:14.713674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:14.714397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:14.714695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:14.716075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:14.728210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 628/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:14.734718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.769200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:14.840563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc776e-6628-4b8c-b3dc-4507b06c9a8e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:14.875226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:14.875855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:14.876125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:14.876389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:14.883763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:14.885198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 22:33:14.896220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:14.897300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:14.898349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 652/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:14.903341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:14.907348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:15.021222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac01a-2ec6-4991-85ce-46d8c7c56bfe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:15.073063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:15.073701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:15.073978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:15.074252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:15.075044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:15.075350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:15.076793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:15.094190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 629/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Mon Aug 1 22:33:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:15.100681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:15.196242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:15.208845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48460bcf-a52e-48f7-ab40-8c489d99f5d0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:15.248335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:15.248933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:15.249207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:15.249469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:15.328663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:15.330070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:15.420111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:15.430854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 653/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Mon Aug 1 22:33:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 22:33:15.437128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:15.505786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:15.545181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a69c67-fef9-4042-9993-60691af4384a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:15.581196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:15.581790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:15.582073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:15.582350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:15.593113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:15.594630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 22:33:15.609963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:15.610960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:15.611909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 22:33:15.611909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 22:33:15.611909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 22:33:15.611909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:15.611909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 630/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:15.617279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:15.718703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22ace3-b847-4277-9876-8e44577c39e3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:15.754025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:15.754631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:15.754903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:15.755172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:15.755874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:15.756178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:15.757584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:15.771167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 654/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Mon Aug 1 22:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:15.777032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:15.878379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22a1f-8bd8-49d1-89ab-aebef379a14f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:15.893264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:15.915046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:15.915680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:15.915959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:15.916232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:15.922726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:15.924205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 22:33:15.935688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:15.936758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Aug 01 22:33:15.937736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 631/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Mon Aug 1 22:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:15.942849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:15.996277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:16.053067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40451-2f41-419b-8424-5a68cd0577ad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:16.089266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:16.089902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:16.090181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:16.090524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:16.091288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:16.091587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:16.093020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281bbfa-ff07-4ea6-a4c8-0f0befd33d30 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:16.105419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 655/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Mon Aug 1 22:33:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:16.114558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:16.232423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c21d-1a02-4b79-849e-fe098a6d3c13 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:16.268230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:16.268833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:16.269115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:16.269388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:16.270214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] RBAC: Adding query filter params (role.id=c512669c168542d587f922137072be5f, scope.project.id=668cc085aa8f430e93314a3a57211a59, user.id=30e6ac9cc1264e109a68bc20fd975623) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:16.270523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:16.271959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4a11-98cc-4750-9a0b-ef385a1e8a15 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:16.278481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 632/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Mon Aug 1 22:33:16 2022] GET /identity/v3/role_assignments?user.id=30e6ac9cc1264e109a68bc20fd975623&scope.project.id=668cc085aa8f430e93314a3a57211a59&role.id=c512669c168542d587f922137072be5f => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:33:18.270775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaa3a4-3430-4681-b8b5-2b5b4b74866e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:18.270775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaa3a4-3430-4681-b8b5-2b5b4b74866e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:18.270775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaa3a4-3430-4681-b8b5-2b5b4b74866e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:18.271787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 656/1288] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:18.278064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:18.278801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:18.278968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:18.301567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:18.302765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:18.334145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:18.376292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 633/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:18.398096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaa3a4-3430-4681-b8b5-2b5b4b74866e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:18.513446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaa3a4-3430-4681-b8b5-2b5b4b74866e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:18.524142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:18.551829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:18.552453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:18.552724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:18.552992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:18.553657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:18.555048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:18.563972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:18.565176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:18.566293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 657/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:33:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:18.571628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:18.674544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55504a9-3d1e-440b-abdc-72750fa0acae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:18.708841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:18.709459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:18.709770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:18.710098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:18.710781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:18.711087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:18.712450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d913f49-8ceb-4ee3-955b-bb394b9de72d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:18.717544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 634/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Aug 1 22:33:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:18.723548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:18.830958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69176c-8728-45e8-a9c7-38e438f4fa74 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:18.865539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:18.866205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:18.866486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:18.866757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:18.867443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:18.867744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:18.869114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff5742-b6a4-419e-adbb-28d5f9149b0d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:18.873587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 658/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Aug 1 22:33:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:20.827957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cce6-45c8-4eb6-96e8-d900119a5079 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:20.828821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cce6-45c8-4eb6-96e8-d900119a5079 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:20.828821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cce6-45c8-4eb6-96e8-d900119a5079 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:20.831015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 635/1293] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:20.838881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e74f-e860-47d0-8bff-18a3b5bb9864 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:20.839335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e74f-e860-47d0-8bff-18a3b5bb9864 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:20.839436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224e74f-e860-47d0-8bff-18a3b5bb9864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:20.868957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5224e74f-e860-47d0-8bff-18a3b5bb9864 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:20.870190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5224e74f-e860-47d0-8bff-18a3b5bb9864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:20.901885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:20.952492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 659/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:20.975461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8cce6-45c8-4eb6-96e8-d900119a5079 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:21.013204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:21.087394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8cce6-45c8-4eb6-96e8-d900119a5079 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:21.122674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:21.123281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:21.123556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:21.123823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:21.124493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:21.125909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc589d10-9118-4f7b-affb-fd708ce8a3e0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:21.142115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 636/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Mon Aug 1 22:33:20 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:33:23.148013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8737c-37ae-45d5-9297-c3bf3cd9e245 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.148973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8737c-37ae-45d5-9297-c3bf3cd9e245 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.148973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8737c-37ae-45d5-9297-c3bf3cd9e245 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.151292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 660/1296] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:23.157608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.158312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.158468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.186067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:23.187269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:23.267487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 637/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:23.292657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8737c-37ae-45d5-9297-c3bf3cd9e245 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:23.413522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8737c-37ae-45d5-9297-c3bf3cd9e245 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:23.448655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:23.449286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.449597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.449866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.450496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:23.451876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:23.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:23.462237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:23.463382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 661/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:33:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:23.472090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:23.492834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:23.578051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c30758-d13c-4ca4-8842-04e83ab4fc1d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:23.620713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:23.621423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.621729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.622000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.622678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:23.622975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:23.624349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:23.629707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 638/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Aug 1 22:33:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:23.636187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:23.670941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:23.747090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53fec6-f953-40fc-ae9a-4ae315382bce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:23.795021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:23.795717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.796140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.796413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.797509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] RBAC: Adding query filter params (interface=public, service_id=cc07f2ad9e2a48298a7869d6c60d22fa, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:23.797993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:23.799647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949fc8-6f55-41c4-b374-e736238d153e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:23.806668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 662/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Mon Aug 1 22:33:23 2022] GET /identity/v3/endpoints?service_id=cc07f2ad9e2a48298a7869d6c60d22fa&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:23.813425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:23.896561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:23.915581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0bea-5e59-4aa7-850a-2e6587944c23 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:23.951867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db309198-8611-4676-b320-35a7682844ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:23.952468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db309198-8611-4676-b320-35a7682844ca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:23.952745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db309198-8611-4676-b320-35a7682844ca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:23.953011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db309198-8611-4676-b320-35a7682844ca None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:23.953688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db309198-8611-4676-b320-35a7682844ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:23.955057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db309198-8611-4676-b320-35a7682844ca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:23.962439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 639/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Mon Aug 1 22:33:23 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:33:25.964578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36266af-c97e-47f9-9fb0-d2d0687919ed None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:25.965147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36266af-c97e-47f9-9fb0-d2d0687919ed None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:25.965147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36266af-c97e-47f9-9fb0-d2d0687919ed None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:25.966530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 663/1302] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:33:25.972709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:25.973426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:25.973631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:26.001090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:33:26.002380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:33:26.072081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 640/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:33:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 22:33:26.094614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36266af-c97e-47f9-9fb0-d2d0687919ed None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:26.200302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:26.211928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36266af-c97e-47f9-9fb0-d2d0687919ed None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:26.248795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:26.249364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:26.249667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:26.249940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:26.250560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:26.251934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 22:33:26.261241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 22:33:26.262638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Aug 01 22:33:26.263769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 664/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Mon Aug 1 22:33:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 22:33:26.270768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:26.347301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:26.372002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681e6ea-5fb6-49d8-9146-f7c2849454dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:26.407657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:26.408270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:26.408553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:26.408827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:26.409508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:33:26.409850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:26.411234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae79cbe-e770-45b5-8ae7-9ecf1907513f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:26.416820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 641/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Mon Aug 1 22:33:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:33:26.422892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:33:26.539521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d263cc-0055-4688-b26f-76cee233cac3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:33:26.574713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:33:26.575300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:33:26.575584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:33:26.575838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:33:26.576539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:33:26.577936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73115976-72cd-4a1d-b9d3-7bb14ff3e6b8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:33:26.602680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 665/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:33:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:33:36.413445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=30 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:33:36.597327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:29.643980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7687a9-bf22-4d72-b4ab-57a5d7ccc5a0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:29.645090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7687a9-bf22-4d72-b4ab-57a5d7ccc5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:29.645634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7687a9-bf22-4d72-b4ab-57a5d7ccc5a0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:29.652975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a941-7c0e-4510-b5b8-01e2e09445dd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:29.654001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a941-7c0e-4510-b5b8-01e2e09445dd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:29.654506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a941-7c0e-4510-b5b8-01e2e09445dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:29.659357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 642/1307] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:29.740400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a909a941-7c0e-4510-b5b8-01e2e09445dd None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:29.742027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a909a941-7c0e-4510-b5b8-01e2e09445dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:29.892288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 666/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:29.942488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:29.942706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:29.943174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:29.945306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 643/1309] 10.222.0.47 () {54 vars in 965 bytes} [Mon Aug 1 22:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:29.952929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5ef1f-520a-4355-96f6-e7a47e8b3f15 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:29.953320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5ef1f-520a-4355-96f6-e7a47e8b3f15 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:29.953899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5ef1f-520a-4355-96f6-e7a47e8b3f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:30.089420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2bb5ef1f-520a-4355-96f6-e7a47e8b3f15 None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:30.090951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bb5ef1f-520a-4355-96f6-e7a47e8b3f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:30.117972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:30.191031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 667/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Mon Aug 1 22:34:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:30.199640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:30.201755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:30.333727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09150e4e-3334-4061-af63-8c8232df0e04 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:30.411102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:30.411728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:30.412219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:30.412504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:30.414154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:30.469976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:30.531124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:30.532497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f573da-033d-4de2-be97-ac4e86b14140 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:30.552594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 644/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:34:32.836722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e423e2-91e7-43ab-9034-02473c1f539f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:32.837331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e423e2-91e7-43ab-9034-02473c1f539f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:32.837331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e423e2-91e7-43ab-9034-02473c1f539f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:32.838809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 668/1312] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:32.844124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508c947-a244-486b-aee5-026f5eeafda5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:32.844567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508c947-a244-486b-aee5-026f5eeafda5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:32.844661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508c947-a244-486b-aee5-026f5eeafda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:32.915824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2508c947-a244-486b-aee5-026f5eeafda5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:32.917037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2508c947-a244-486b-aee5-026f5eeafda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:33.028493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 645/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:33.089998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:33.090327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:33.090838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:33.093057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 669/1314] 10.222.0.47 () {54 vars in 965 bytes} [Mon Aug 1 22:34:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:33.102308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc740d5-b025-4003-92dd-894c054a16da None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:33.102566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc740d5-b025-4003-92dd-894c054a16da None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:33.102867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc740d5-b025-4003-92dd-894c054a16da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:33.115657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:33.180150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2dc740d5-b025-4003-92dd-894c054a16da None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:33.181347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2dc740d5-b025-4003-92dd-894c054a16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:33.258796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 646/1315] 10.222.0.47 () {58 vars in 1062 bytes} [Mon Aug 1 22:34:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:33.268625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:33.270256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:33.391105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc646-7a3c-43bd-8866-0e49ee827d86 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:33.471647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:33.474581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:33.475506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:33.475798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:33.476471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:33.478178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:33.604294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:33.605712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf1fe7-2ae8-4950-ad3d-4d8501fbf7e1 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:33.627566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 670/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:36.135783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:36.136309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:36.136421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:36.137842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 647/1317] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:36.143208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f831-f07d-485e-a8aa-e227f63c8f35 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:36.143593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f831-f07d-485e-a8aa-e227f63c8f35 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:36.143746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f831-f07d-485e-a8aa-e227f63c8f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:36.226126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9ce0f831-f07d-485e-a8aa-e227f63c8f35 None None] MFA Rules not processed for user `e7877688e78b41359eb7ae4a89ec65df`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:36.227351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9ce0f831-f07d-485e-a8aa-e227f63c8f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:36.284817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:36.333200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 671/1318] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:36.373247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:36.374848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:36.484749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b257957-1e31-4a35-947b-cd27f0ac5172 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:36.487561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:36.562438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:36.562984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:36.563260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:36.563528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:36.565146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:36.678542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:36.679988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8af70-35d1-47b2-bbf0-ef7f2b956b77 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:36.698711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 648/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:36 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:39.422699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:39.423230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:39.423230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:39.425400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 672/1320] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:39.431754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9eaab-62c9-4236-9351-572ff7dd1184 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:39.432212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9eaab-62c9-4236-9351-572ff7dd1184 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:39.432306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9eaab-62c9-4236-9351-572ff7dd1184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:39.475537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:39.501144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-40c9eaab-62c9-4236-9351-572ff7dd1184 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:39.502357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40c9eaab-62c9-4236-9351-572ff7dd1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:39.607261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 649/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:34:39.649139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:39.651567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:39.781012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4f3c0-0cd3-4594-ba58-9dc209d042ad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:39.861366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:39.862019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:39.862299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:39.862573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:39.864244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:39.872426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:39.993930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:39.995365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38171146-a52d-4e7a-972e-26815956342e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:40.014187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 673/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:42.736065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:42.736635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:42.736635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:42.738046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 650/1323] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:42.743656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65fafd-5e8a-45b8-ac73-a7f6a92e1357 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:42.744089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65fafd-5e8a-45b8-ac73-a7f6a92e1357 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:42.744394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65fafd-5e8a-45b8-ac73-a7f6a92e1357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:42.811536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc65fafd-5e8a-45b8-ac73-a7f6a92e1357 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:42.812752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc65fafd-5e8a-45b8-ac73-a7f6a92e1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:42.878869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:42.925581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 674/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:42.962680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:42.964408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:43.076291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2013a-eaa1-4af3-bade-496b9eb8ed1b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:43.088609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:43.153095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:43.153759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:43.154037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:43.154329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:43.155967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:43.269733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:43.271140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c37f4-8383-47fd-a12f-afac697d3b81 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:43.290223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 651/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:45.349733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:45.350306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:45.350306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:45.351562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 675/1326] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:45.356910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babdd6d-9ff4-4b45-ab3b-57425a5222a4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:45.357321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babdd6d-9ff4-4b45-ab3b-57425a5222a4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:45.357547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babdd6d-9ff4-4b45-ab3b-57425a5222a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:45.406670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:45.424821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3babdd6d-9ff4-4b45-ab3b-57425a5222a4 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:45.426058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3babdd6d-9ff4-4b45-ab3b-57425a5222a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:45.528278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 652/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:45.568063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:45.569593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:45.680576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886a732-1957-4bc4-a45b-c85f0100bf75 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:45.761339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:45.761999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:45.762274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:45.762547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:45.764233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:45.782125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:45.886124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:45.887577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750bc8d-bf7a-4030-80ad-961cb0a1e5d2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:45.907188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 676/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:47.952494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:47.953056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:47.953056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:47.954585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 653/1329] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:47.960084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fac2-d40c-43d5-8c10-485eadbcc497 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:47.960516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fac2-d40c-43d5-8c10-485eadbcc497 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:47.960748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fac2-d40c-43d5-8c10-485eadbcc497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:48.032722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3464fac2-d40c-43d5-8c10-485eadbcc497 None None] MFA Rules not processed for user `e7877688e78b41359eb7ae4a89ec65df`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:48.033979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3464fac2-d40c-43d5-8c10-485eadbcc497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:48.106664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:48.141018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 677/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:48.178566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:48.180094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:48.289134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63162a5f-76c2-471d-9083-92e9ef505961 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:48.309736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:48.366252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:48.366874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:48.367149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:48.367420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:48.369423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:48.494159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:48.495625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45610fa9-de74-4f20-8656-3053defe62d3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:48.521390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 654/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:50.768887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:50.769935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:50.769935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:50.772036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 678/1332] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:50.780179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409607e-edc9-4af7-885f-ef0c5325b084 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:50.780627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409607e-edc9-4af7-885f-ef0c5325b084 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:50.780750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409607e-edc9-4af7-885f-ef0c5325b084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:50.856001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:50.859294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d409607e-edc9-4af7-885f-ef0c5325b084 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:50.860778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d409607e-edc9-4af7-885f-ef0c5325b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:50.971906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 655/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:51.010768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:51.012298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:51.116353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb23592-b009-41ff-aee7-c420b8fc69f5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:51.193425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:51.194063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:51.194334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:51.194600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:51.196197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:51.224063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:51.310543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:51.311969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590fc7-f055-4099-b490-17b8e036fb41 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:51.330229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 679/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:53.395329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:53.396102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:53.396244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:53.398499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 656/1335] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:53.404852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f9a-aa78-465d-b11f-f0671230e32f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:53.405550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f9a-aa78-465d-b11f-f0671230e32f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:53.405783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f9a-aa78-465d-b11f-f0671230e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:53.484743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e0ad4f9a-aa78-465d-b11f-f0671230e32f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:53.485999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0ad4f9a-aa78-465d-b11f-f0671230e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:53.562928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:53.585408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 680/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:53.628872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:53.631581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:53.749144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae13b80-fe7f-43c8-aae4-c4f396348a28 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:53.779923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:53.825753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:53.826380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:53.826658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:53.826927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:53.828550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:53.951850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:53.953300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbfb72-abf0-4df1-ab63-2fe4e6d8c6ca service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:53.972486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 657/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:34:56.468842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:56.470143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:56.470143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:56.472300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 681/1338] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:34:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:34:56.480334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3f3-b7ad-4844-827b-153a44a9d90a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:56.480758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3f3-b7ad-4844-827b-153a44a9d90a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:56.480891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3f3-b7ad-4844-827b-153a44a9d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:56.556090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c5d5e3f3-b7ad-4844-827b-153a44a9d90a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:34:56.557294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c5d5e3f3-b7ad-4844-827b-153a44a9d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:56.566467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:56.659297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 658/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:34:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:34:56.695229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:34:56.696734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:56.814933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244a9d2-2c25-49fb-b244-71a144568b05 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:34:56.896699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:34:56.897310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:34:56.897604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:34:56.897888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:34:56.899503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:34:56.934961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:34:57.018376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:34:57.019842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5633ccb-906a-4998-aa2d-820e2df0a45a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:34:57.039120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 682/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:34:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:35:01.123068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:01.123638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:01.123638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:01.125014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 659/1341] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:01.131316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196d99-c278-430f-b422-9d6f3ad4c33b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:01.131783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196d99-c278-430f-b422-9d6f3ad4c33b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:01.131974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196d99-c278-430f-b422-9d6f3ad4c33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:01.206585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-86196d99-c278-430f-b422-9d6f3ad4c33b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:01.207807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86196d99-c278-430f-b422-9d6f3ad4c33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:01.300757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:01.312435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 683/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:01.351529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:01.353166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:01.464469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f73ae3-7905-49a8-85cb-6bf0b6802b14 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:01.504325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:01.540137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:01.540765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:01.541040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:01.541308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:01.543002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:01.658540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:01.660003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbb473-b312-4ecd-bc6d-087b621e0e4c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:01.678734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 660/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:04.292099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:04.292649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:04.292649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:04.293979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 684/1344] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:04.299872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f829-8e57-4b97-8ecf-749860cbec37 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:04.300603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f829-8e57-4b97-8ecf-749860cbec37 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:04.300603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f829-8e57-4b97-8ecf-749860cbec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:04.373174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-04b6f829-8e57-4b97-8ecf-749860cbec37 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:04.374667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-04b6f829-8e57-4b97-8ecf-749860cbec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:04.392839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:04.475624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 661/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:04.512991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:04.514535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:04.623428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb7b69-6d9b-4773-9d30-9e3d1a277e18 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:04.698684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:04.699306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:04.699584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:04.699855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:04.701480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:04.751308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:04.825931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:04.827367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b511d-c433-4587-8957-17aaea5b0d7b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:04.847552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 685/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:08.769321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:08.769978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:08.769978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:08.771430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 662/1347] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:08.776549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f059d-f1fc-4641-8602-5cade965afa7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:08.776992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f059d-f1fc-4641-8602-5cade965afa7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:08.777104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f059d-f1fc-4641-8602-5cade965afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:08.842786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-005f059d-f1fc-4641-8602-5cade965afa7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:08.844269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-005f059d-f1fc-4641-8602-5cade965afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:08.939813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:08.945700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 686/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:08.980943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:08.982696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:09.110122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb852-41ab-498d-9560-7b9dca9f636d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:09.161027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:09.188503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:09.188942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:09.189219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:09.189488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:09.191230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:09.307777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:09.309330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6d7c-e555-4424-a4a1-3076bec4fee1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:09.329298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 663/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:14.228008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f619-3c32-4cd5-a0d5-3766ea2a6b04 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:14.228495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f619-3c32-4cd5-a0d5-3766ea2a6b04 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:14.228641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f619-3c32-4cd5-a0d5-3766ea2a6b04 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:14.230238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 687/1350] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:14.236239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad6416-99c2-4ea2-b91e-17b39ac5c22e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:14.237235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad6416-99c2-4ea2-b91e-17b39ac5c22e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:14.237742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad6416-99c2-4ea2-b91e-17b39ac5c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:14.369452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-06ad6416-99c2-4ea2-b91e-17b39ac5c22e None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:14.370840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06ad6416-99c2-4ea2-b91e-17b39ac5c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:14.391681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:14.489670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 664/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:35:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:18.718882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfce4-138d-4089-9e69-6b3461359b20 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:18.719415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfce4-138d-4089-9e69-6b3461359b20 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:18.719415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfce4-138d-4089-9e69-6b3461359b20 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:18.720887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 688/1352] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:18.726830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cd4d-7e30-49ed-9194-a5e5a40a8ad2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:18.727480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cd4d-7e30-49ed-9194-a5e5a40a8ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:18.727480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cd4d-7e30-49ed-9194-a5e5a40a8ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:18.811338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5b36cd4d-7e30-49ed-9194-a5e5a40a8ad2 None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:18.812584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b36cd4d-7e30-49ed-9194-a5e5a40a8ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:18.901357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:18.942122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 665/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:35:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:35:20.049019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88581d-492b-4c9f-8db9-abaa2e7f5b21 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:20.049631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88581d-492b-4c9f-8db9-abaa2e7f5b21 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:20.049631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88581d-492b-4c9f-8db9-abaa2e7f5b21 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:20.051130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 689/1354] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:20.056150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d698a7-17f2-4de7-8f95-74f4accaccac None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:20.056559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d698a7-17f2-4de7-8f95-74f4accaccac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:20.056701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d698a7-17f2-4de7-8f95-74f4accaccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:20.135450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c6d698a7-17f2-4de7-8f95-74f4accaccac None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:20.136659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c6d698a7-17f2-4de7-8f95-74f4accaccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:20.247280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 666/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:35:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:23.591298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c816b-6e2e-4580-87e5-063cf07ef5ff None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:23.591857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c816b-6e2e-4580-87e5-063cf07ef5ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:23.591857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c816b-6e2e-4580-87e5-063cf07ef5ff None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:23.597550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 690/1356] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:23.599021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b84b-b840-4650-be85-aa1b19a2ee04 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:23.599841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b84b-b840-4650-be85-aa1b19a2ee04 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:23.600322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b84b-b840-4650-be85-aa1b19a2ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:23.639295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:23.672671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8dc3b84b-b840-4650-be85-aa1b19a2ee04 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:23.673909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8dc3b84b-b840-4650-be85-aa1b19a2ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:23.776434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 667/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:35:23.831114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:23.831854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:23.832356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:23.834415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 691/1358] 10.222.0.47 () {54 vars in 962 bytes} [Mon Aug 1 22:35:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:23.843313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0424d2-dee2-4c42-b85d-06199de9c639 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:23.845295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0424d2-dee2-4c42-b85d-06199de9c639 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:23.846255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0424d2-dee2-4c42-b85d-06199de9c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:23.930700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1b0424d2-dee2-4c42-b85d-06199de9c639 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:23.931939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b0424d2-dee2-4c42-b85d-06199de9c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:24.111364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:24.184396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 668/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Aug 1 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:24.192098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:24.193832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:24.340434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f52f4-272e-4d2c-98f1-0edb36ca1156 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:24.416792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:24.458262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:24.458886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:24.459155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:24.459417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:24.461012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:24.558671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:24.560653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79621f-69c5-44cc-a57a-a22f3fc6a948 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:24.580041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 692/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:26.768685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:26.769228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:26.769228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:26.772091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 669/1361] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:26.776082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4189d2-310d-4b84-9d3c-0f72525df2d7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:26.776579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4189d2-310d-4b84-9d3c-0f72525df2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:26.776579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4189d2-310d-4b84-9d3c-0f72525df2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:26.845406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2d4189d2-310d-4b84-9d3c-0f72525df2d7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:26.846650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2d4189d2-310d-4b84-9d3c-0f72525df2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:26.891909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:26.981610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 693/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:27.021674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:27.023606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:27.216312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39c11-9a6f-416f-8300-7cdb0e16f889 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:27.226603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:27.343365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:27.344218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:27.344539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:27.344869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:27.346632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:27.445697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:27.447341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cde7e2-5a8b-468a-86b9-b52f5abb584d service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:27.466454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 670/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:29.809597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac47bc-b072-4571-aef0-f296746f5a58 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:29.810190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac47bc-b072-4571-aef0-f296746f5a58 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:29.810190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac47bc-b072-4571-aef0-f296746f5a58 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:29.811708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 694/1364] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:29.817056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befce5dd-8251-452f-8249-f1d2fce6ad13 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:29.817528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befce5dd-8251-452f-8249-f1d2fce6ad13 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:29.817808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befce5dd-8251-452f-8249-f1d2fce6ad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:29.834180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:29.888836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-befce5dd-8251-452f-8249-f1d2fce6ad13 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:29.890063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-befce5dd-8251-452f-8249-f1d2fce6ad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:29.990310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 671/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:30.035595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:30.036172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:30.036548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:30.038183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 695/1366] 10.222.0.47 () {54 vars in 962 bytes} [Mon Aug 1 22:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:30.043723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ddac-d20c-4cd1-927c-84cd5286d449 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:30.043974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ddac-d20c-4cd1-927c-84cd5286d449 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:30.044252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ddac-d20c-4cd1-927c-84cd5286d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:30.126541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1a12ddac-d20c-4cd1-927c-84cd5286d449 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:30.127924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a12ddac-d20c-4cd1-927c-84cd5286d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:30.165599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:30.269730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 672/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Aug 1 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:30.277310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:30.279040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:30.442213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee2476-3452-4293-ae9a-d86919171ae8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:30.463308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:30.562649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:30.563277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:30.563558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:30.563857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:30.565525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:30.665750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:30.667350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443836-4ff7-45d8-a6ef-3d65900932d2 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:30.687619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 696/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:31.557995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a984f-f657-4dfa-932a-98ab04cd9205 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:31.558549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a984f-f657-4dfa-932a-98ab04cd9205 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:31.558549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a984f-f657-4dfa-932a-98ab04cd9205 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:31.560160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 673/1369] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:31.565609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49a50-20cb-4479-83fa-315ee12d1eb9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:31.566064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49a50-20cb-4479-83fa-315ee12d1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:31.566064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49a50-20cb-4479-83fa-315ee12d1eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:31.600424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:31.638660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-07b49a50-20cb-4479-83fa-315ee12d1eb9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:31.639914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07b49a50-20cb-4479-83fa-315ee12d1eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:31.742244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 697/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:35:32.093334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:32.093649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:32.093910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:32.095475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 674/1371] 10.222.0.47 () {54 vars in 964 bytes} [Mon Aug 1 22:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:32.100536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8931f9-ccbe-48fc-95f2-0d5f4741e36d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:32.100968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8931f9-ccbe-48fc-95f2-0d5f4741e36d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:32.101336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8931f9-ccbe-48fc-95f2-0d5f4741e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:32.184225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b8931f9-ccbe-48fc-95f2-0d5f4741e36d None None] MFA Rules not processed for user `84a57435ddd44fe79517001c2d91eec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:32.185800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b8931f9-ccbe-48fc-95f2-0d5f4741e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:32.238817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:32.272331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 698/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Mon Aug 1 22:35:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:32.280250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:32.281868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:32.406897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9478c48-2675-4014-a6e8-57852fb6ef0f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:32.452201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:32.488435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:32.489228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:32.489610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:32.489991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:32.491893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:32.612597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:32.614104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104692-7e16-4fad-8153-7c67b89e905c service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:32.636776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 675/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:35:32.770162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:32.770162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:32.770162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:32.774604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 699/1374] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:32.776274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c3c-7b95-4bb1-9ce0-6a4641ed6d34 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:32.776429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c3c-7b95-4bb1-9ce0-6a4641ed6d34 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:32.776736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c3c-7b95-4bb1-9ce0-6a4641ed6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:32.851598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eef15c3c-7b95-4bb1-9ce0-6a4641ed6d34 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:32.852787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eef15c3c-7b95-4bb1-9ce0-6a4641ed6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:32.875642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:32.961145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 676/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:33.009799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:33.011517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:33.164613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecbafa-bb7f-4d01-92cd-1b2c063e0820 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:33.227359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:33.290360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:33.291243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:33.291522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:33.291820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:33.293455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:33.389002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:33.390630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91550280-ef3d-42bb-9b1a-648c0b326957 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:33.409240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 700/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:35.435945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:35.436407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:35.436496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:35.437915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 677/1377] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:35.443400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bcb23-34fe-487c-8132-3ee2949da064 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:35.443862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bcb23-34fe-487c-8132-3ee2949da064 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:35.443958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bcb23-34fe-487c-8132-3ee2949da064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:35.526374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc6bcb23-34fe-487c-8132-3ee2949da064 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:35.527616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc6bcb23-34fe-487c-8132-3ee2949da064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:35.534630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:35.653730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 701/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:35.704729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:35.706658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:35.857691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded997e5-a642-4f10-8fbe-ec31e0f26507 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:35.861368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:35.971183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:35.971813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:35.972091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:35.972358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:35.976162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:36.093684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:36.095670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd8c7-ba1b-4092-9805-32327bde7a46 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:36.116153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 678/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:38.183497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:38.184057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:38.184057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:38.185647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 702/1380] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:38.191005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c3f-d5dc-4e45-b0ff-c4e0bc6c0b5a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:38.191483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c3f-d5dc-4e45-b0ff-c4e0bc6c0b5a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:38.191566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c3f-d5dc-4e45-b0ff-c4e0bc6c0b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:38.203606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:38.262450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0d8a5c3f-d5dc-4e45-b0ff-c4e0bc6c0b5a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:38.263727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d8a5c3f-d5dc-4e45-b0ff-c4e0bc6c0b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:38.369959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 679/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:38.411301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:38.413524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:38.559034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:38.570496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd8aa-5b81-451d-8df0-1b7e825f519d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:38.689990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:38.690556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:38.690834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:38.691104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:38.692726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:38.792812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:38.794456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42456056-4891-4224-a99b-dd1eed1fa530 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:38.806070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:38.827665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 703/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:40.875258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:40.875922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:40.876082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:40.878040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 680/1383] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:40.883815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e61cd-4662-47a5-95a0-18b4aa341c70 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:40.884451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e61cd-4662-47a5-95a0-18b4aa341c70 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:40.884451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e61cd-4662-47a5-95a0-18b4aa341c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:40.960815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0b0e61cd-4662-47a5-95a0-18b4aa341c70 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:40.962242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0b0e61cd-4662-47a5-95a0-18b4aa341c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:41.075899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 704/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:41.119132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:41.120905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:41.223493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:41.305073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd840f68-f082-4cd0-8828-069cb28a8134 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:41.428876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:41.429640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:41.429919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:41.430187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:41.431853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:41.490688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:41.534176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:41.535943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b66133-05e8-4e71-8d5b-0855d2f7e306 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:41.555239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 681/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:43.607009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:43.607700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:43.607700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:43.608898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 705/1386] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:43.614523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9742a-96df-4bda-a33e-f69ddc8e9899 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:43.615105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9742a-96df-4bda-a33e-f69ddc8e9899 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:43.615236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9742a-96df-4bda-a33e-f69ddc8e9899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:43.691900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-81f9742a-96df-4bda-a33e-f69ddc8e9899 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:43.693756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-81f9742a-96df-4bda-a33e-f69ddc8e9899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:43.812631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 682/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:43.860980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:43.862638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:43.946774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:44.026401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38711c80-e199-4efb-8666-0e4eef3d0321 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:44.178352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:44.179149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:44.179514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:44.179891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:44.182200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:44.228612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:44.285584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:44.287359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c7ede-56ea-41e6-9ddb-25ae77d7a565 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:44.307283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 706/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:46.344938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:46.345475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:46.345475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:46.346802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 683/1389] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:46.352473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c1e6-e347-404c-a151-42d42e3cd7dc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:46.352944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c1e6-e347-404c-a151-42d42e3cd7dc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:46.353037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174c1e6-e347-404c-a151-42d42e3cd7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:46.423833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7174c1e6-e347-404c-a151-42d42e3cd7dc None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:46.425046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7174c1e6-e347-404c-a151-42d42e3cd7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:46.529664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 707/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:46.574907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:46.576582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:46.608702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:46.728313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2911-efbb-4e41-9c5b-6b21a8dc5534 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:46.830942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:46.843252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:46.843881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:46.844177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:46.844431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:46.846426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:46.936581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:46.942482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d133c-60bc-42bf-8aa8-fdef5590f962 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:46.961174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 684/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:47.583991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf50831-acf4-41b2-aecc-7e3d3493b9b0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:47.584559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf50831-acf4-41b2-aecc-7e3d3493b9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:47.584559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf50831-acf4-41b2-aecc-7e3d3493b9b0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:47.586718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 708/1392] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:47.591108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fe849-0272-4f41-8b3c-b96550c8d06f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:47.591555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fe849-0272-4f41-8b3c-b96550c8d06f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:47.591651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fe849-0272-4f41-8b3c-b96550c8d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:47.660915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9f1fe849-0272-4f41-8b3c-b96550c8d06f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:47.662149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f1fe849-0272-4f41-8b3c-b96550c8d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:47.722429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:47.761327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 685/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:47.815399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:47.815681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:47.815949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:47.817374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 709/1394] 10.222.0.47 () {54 vars in 966 bytes} [Mon Aug 1 22:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:47.822549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9a399-3154-4df2-ab8a-17a8fbe223f4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:47.822840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9a399-3154-4df2-ab8a-17a8fbe223f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:47.823140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9a399-3154-4df2-ab8a-17a8fbe223f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:47.899208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dfe9a399-3154-4df2-ab8a-17a8fbe223f4 None None] MFA Rules not processed for user `0f05345f84a448f3842f4e91d9f0ac67`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:47.900376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dfe9a399-3154-4df2-ab8a-17a8fbe223f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:47.979308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 686/1395] 10.222.0.47 () {58 vars in 1063 bytes} [Mon Aug 1 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:47.986933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:47.988507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:48.003218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:48.096068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9371638-0545-45d1-b070-fb55a430bebb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:48.170943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:48.171565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:48.171837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:48.172088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:48.173729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:48.240083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:48.281727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:48.283236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a009-84bb-4f6a-bea3-38b661a82bda service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:48.301060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 710/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:49.003856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90424309-da12-416e-871c-373bcdeca359 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.004445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90424309-da12-416e-871c-373bcdeca359 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.004445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90424309-da12-416e-871c-373bcdeca359 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.005960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 687/1397] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:49.011674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.012192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.012192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.056755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27f9a-d85f-4fe1-beb0-d5c75fcddc4d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.057122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27f9a-d85f-4fe1-beb0-d5c75fcddc4d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.057485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27f9a-d85f-4fe1-beb0-d5c75fcddc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.098094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:49.099289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.132508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-77c27f9a-d85f-4fe1-beb0-d5c75fcddc4d None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:49.133775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77c27f9a-d85f-4fe1-beb0-d5c75fcddc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.142151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:49.210084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 711/1398] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:49.229709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 688/1399] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:35:49.260848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:49.262196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.276822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837c75-111b-453e-81ef-b3636d3a2b3b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.277108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837c75-111b-453e-81ef-b3636d3a2b3b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.277366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837c75-111b-453e-81ef-b3636d3a2b3b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.278778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 689/1400] 10.222.0.47 () {54 vars in 943 bytes} [Mon Aug 1 22:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:35:49.285224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.285503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.285818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.296076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:49.364957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] MFA Rules not processed for user `223d4641075e4b6489e3ee1e4fbdce36`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:49.366250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.390423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1e70e-d1d6-4304-810b-b9a99a74bf4a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:49.455712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:49.481155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 690/1401] 10.222.0.47 () {58 vars in 1040 bytes} [Mon Aug 1 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:35:49.490857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:49.500750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965bf2a-cb39-44b0-967c-61b9310b27c8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:49.512935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:49.513504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.513815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.514091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.515740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.535762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:49.609361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:49.610013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:49.610373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:49.610642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:49.612486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:49.616226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:49.617876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190905e-695b-4e8f-833c-43baff7b2ee9 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:49.636874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 712/1402] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:49.705597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:49.707330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f0a10-bb1d-45b4-a15a-b07864811e3f service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:49.717227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 691/1403] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:35:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 22:35:51.718767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:51.719372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:51.719372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:51.720821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 713/1404] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:51.726035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c6e-74ef-43ec-bf55-d9525ae1eba9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:51.726539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c6e-74ef-43ec-bf55-d9525ae1eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:51.726643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9c6e-74ef-43ec-bf55-d9525ae1eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:51.738322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:51.802104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8f5f9c6e-74ef-43ec-bf55-d9525ae1eba9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:51.803323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f5f9c6e-74ef-43ec-bf55-d9525ae1eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:51.906102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 692/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:51.954602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:51.956207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:52.123523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:52.127250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a903c8e-eb30-47be-ad33-ebf77df95cd4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:52.255376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:52.256029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:52.256301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:52.256573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:52.258530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:52.360862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:52.362564 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc15c-96b6-49b6-9d55-646e19f5ba61 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:52.381429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 714/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:35:54.415051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:54.415617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:54.415617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:54.416960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 693/1407] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:35:54.422339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743955-82dc-4512-a636-e0f7dbc521de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:54.422786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743955-82dc-4512-a636-e0f7dbc521de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:54.422899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743955-82dc-4512-a636-e0f7dbc521de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:54.427571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:54.495732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2e743955-82dc-4512-a636-e0f7dbc521de None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:35:54.497044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2e743955-82dc-4512-a636-e0f7dbc521de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:54.607053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 715/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:35:54.652089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:35:54.653686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:54.750373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:54.884861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f830f96-563c-4433-b6dd-d77b2450cb1b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:35:55.015991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:35:55.016385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:35:55.016664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:35:55.016932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:35:55.018750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:35:55.077693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:35:55.122781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:35:55.124847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318fa30-0137-4b25-b475-44dd2ab0ac28 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:35:55.146550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 694/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:04.605435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=64 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:05.143551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=23 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:11.983080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9ad02-0ab5-4ae8-9009-ddec97e89013 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:11.983688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9ad02-0ab5-4ae8-9009-ddec97e89013 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:11.983688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9ad02-0ab5-4ae8-9009-ddec97e89013 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:11.985130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 716/1410] 10.222.0.47 () {54 vars in 918 bytes} [Mon Aug 1 22:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:11.990471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecfd0-0e20-48fd-a5b2-d64ec2168d86 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:11.990953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecfd0-0e20-48fd-a5b2-d64ec2168d86 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:11.991054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecfd0-0e20-48fd-a5b2-d64ec2168d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:12.067118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b47ecfd0-0e20-48fd-a5b2-d64ec2168d86 None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:12.068244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b47ecfd0-0e20-48fd-a5b2-d64ec2168d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:12.178241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 695/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Aug 1 22:36:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:12.194711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afecec-0c96-4825-996f-c85b737e0817 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:12.194960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afecec-0c96-4825-996f-c85b737e0817 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:12.195238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afecec-0c96-4825-996f-c85b737e0817 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:12.196597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 717/1412] 10.222.0.47 () {54 vars in 918 bytes} [Mon Aug 1 22:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:12.201586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c55ea-3437-40c1-b7d8-96c272442bcd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:12.201890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c55ea-3437-40c1-b7d8-96c272442bcd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:12.202167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c55ea-3437-40c1-b7d8-96c272442bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:12.276776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:12.279723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8c6c55ea-3437-40c1-b7d8-96c272442bcd None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:12.281482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c6c55ea-3437-40c1-b7d8-96c272442bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:12.400360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 696/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Aug 1 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:36:14.876247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:14.877176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:14.877176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:14.879391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:14.885806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06ce8-ad2f-4d24-af2a-c9343d64af0e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:14.886251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06ce8-ad2f-4d24-af2a-c9343d64af0e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:14.886357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad06ce8-ad2f-4d24-af2a-c9343d64af0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:14.959463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cad06ce8-ad2f-4d24-af2a-c9343d64af0e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:14.960662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cad06ce8-ad2f-4d24-af2a-c9343d64af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:15.024144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:15.068546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:15.126701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:15.128176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:15.293334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e243f9-3e36-416e-9aea-44ba16aa6ad2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:15.381652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:15.425404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:15.426051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:15.426327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:15.426601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:15.428229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:15.534827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:15.536433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec1e0-0362-493a-82a5-763a957d57b0 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:15.555907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 719/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:16.465507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:16.466425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:16.466425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:16.467858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:16.473105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25db84-bd8c-4309-8efe-e6c76fd5ba4c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:16.473583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25db84-bd8c-4309-8efe-e6c76fd5ba4c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:16.473700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25db84-bd8c-4309-8efe-e6c76fd5ba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:16.558969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3a25db84-bd8c-4309-8efe-e6c76fd5ba4c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:16.560170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3a25db84-bd8c-4309-8efe-e6c76fd5ba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:16.587968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:16.661758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:16.821228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:16.823591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:16.992527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37f3c-f02c-4810-b577-d3e75a98f4a8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:17.033951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:17.111860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:17.112397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:17.112668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:17.112933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:17.114609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:17.210275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:17.212430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772a6c8-0592-4659-8421-22a9fdbd53fd service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:17.236961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 699/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:18.139914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:18.140822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:18.140822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:18.143027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:18.151722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273bd3-2ce6-4f39-94a2-be07d1f208bb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:18.152163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273bd3-2ce6-4f39-94a2-be07d1f208bb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:18.152285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273bd3-2ce6-4f39-94a2-be07d1f208bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:18.209739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:18.234303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-63273bd3-2ce6-4f39-94a2-be07d1f208bb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:18.235499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-63273bd3-2ce6-4f39-94a2-be07d1f208bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:18.340567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:18.491394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:18.493344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:18.655439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15125606-b05a-476c-82b4-092a5f751f89 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:18.667942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:18.773152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:18.773824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:18.774097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:18.774360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:18.775975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:18.871386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:18.872995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20faa76-6bd6-47fb-8cde-30368346eb9e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:18.891626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 722/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:19.774588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:19.775182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:19.775182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:19.776508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:36:19.781683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f72a3e-36af-43b1-9ab1-d3af3d4bba20 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:19.782115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f72a3e-36af-43b1-9ab1-d3af3d4bba20 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:19.782210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f72a3e-36af-43b1-9ab1-d3af3d4bba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:19.803054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:19.869766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-38f72a3e-36af-43b1-9ab1-d3af3d4bba20 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:19.870965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38f72a3e-36af-43b1-9ab1-d3af3d4bba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:19.974456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:20.127836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:20.129278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:20.251667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:20.289750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404ef1-d419-4700-9343-a923b7654822 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:20.425587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:20.426125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:20.426407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:20.426680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:20.428313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:20.508978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:20.523389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:20.524990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be1e4-f2c7-4809-9c59-b157d4a1021c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:20.543334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 702/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:21.420733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:21.421301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:21.421301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:21.422721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:21.428035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6667dc-82aa-495a-a65a-7c927507e98d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:21.428525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6667dc-82aa-495a-a65a-7c927507e98d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:21.428618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6667dc-82aa-495a-a65a-7c927507e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:21.508377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4d6667dc-82aa-495a-a65a-7c927507e98d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:21.509660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d6667dc-82aa-495a-a65a-7c927507e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:21.612655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:21.763336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:21.764834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:21.852019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:21.924557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43ba63-5c38-45a6-8942-e152d1bad101 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:22.050376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:22.050998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:22.051276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:22.051562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:22.053146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:22.113206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:22.158662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:22.160265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81640-6254-4e85-af39-4dbcf3ce62e3 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:22.179109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 725/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:22.997900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:22.998508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:22.998508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:22.999822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:23.007200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a051d3-69b2-4681-8186-2e9d7c49a57f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:23.007678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a051d3-69b2-4681-8186-2e9d7c49a57f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:23.007794 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a051d3-69b2-4681-8186-2e9d7c49a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:23.090101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f5a051d3-69b2-4681-8186-2e9d7c49a57f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:23.091671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f5a051d3-69b2-4681-8186-2e9d7c49a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:23.199536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:23.348740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:23.350193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:23.408932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:23.529308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7ad96-6b18-41ad-8167-e25982fb9691 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:23.649255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:23.649817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:23.650093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:23.650362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:23.651971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:23.658840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:23.756100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:23.758169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e24d-2f4a-4ab8-a69c-fe8ab222f19c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:23.779723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 705/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:24.584220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:24.584830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:24.584830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:24.586929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:36:24.595744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524767-49d5-4f0b-8363-0f49426a8b04 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:24.596179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524767-49d5-4f0b-8363-0f49426a8b04 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:24.596383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524767-49d5-4f0b-8363-0f49426a8b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:24.679514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-99524767-49d5-4f0b-8363-0f49426a8b04 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:24.680718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-99524767-49d5-4f0b-8363-0f49426a8b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:24.753415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:24.784105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:24.936856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:24.938651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:24.957574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:25.088057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3c99-03ad-429a-a8d8-f58c86aaae26 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:25.183375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:25.210442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:25.211057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:25.211357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:25.211620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:25.213230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:25.431830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:25.433412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e799a8-d7b6-4c96-9149-b9a5538e7c49 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:25.452323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 728/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:26.269938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:26.270665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:26.270665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:26.272640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:26.280553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a666-1008-4e3f-925b-db424da6daec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:26.281114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a666-1008-4e3f-925b-db424da6daec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:26.281240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089a666-1008-4e3f-925b-db424da6daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:26.365218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2089a666-1008-4e3f-925b-db424da6daec None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:26.366440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2089a666-1008-4e3f-925b-db424da6daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:26.408270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:26.463567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:26.613041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:26.614481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:26.783683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75736ba3-873f-4621-a846-b9e9b31e169b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:26.841992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:26.903202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:26.903740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:26.904025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:26.904295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:26.906072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:27.002150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:27.003861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320e81b-4b1d-4f00-8c18-12a4addacb61 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:27.022547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 708/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:27.818849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:27.819495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:27.819495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:27.820751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:27.825841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9793-af79-4187-b0ee-24c19bdd6405 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:27.826323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9793-af79-4187-b0ee-24c19bdd6405 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:27.826427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9793-af79-4187-b0ee-24c19bdd6405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:27.908732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a05e9793-af79-4187-b0ee-24c19bdd6405 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:27.910006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a05e9793-af79-4187-b0ee-24c19bdd6405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:27.912072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:28.015846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:28.168713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:28.171001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:28.371890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498c2f7-b3ce-48d1-82ee-c9977c23dc00 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:28.409310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:28.512194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:28.512836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:28.513081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:28.513373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:28.515021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:28.609534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:28.611181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f56a-24fa-4d21-82eb-e5bb6c1c7e41 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:28.630212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 731/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:36:30.553098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:30.553712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:30.553712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:30.555802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:30.564645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d589dd-9157-4a01-b176-0c05d01d79b5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:30.565178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d589dd-9157-4a01-b176-0c05d01d79b5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:30.565178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d589dd-9157-4a01-b176-0c05d01d79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:30.608744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:30.634405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-95d589dd-9157-4a01-b176-0c05d01d79b5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:30.635614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-95d589dd-9157-4a01-b176-0c05d01d79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:30.761322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:30.804768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:30.806308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:30.913157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d682599-7269-497b-9faa-fa4b6f537ff2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:30.946447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:30.988588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:30.989205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:30.989486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:30.989777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:30.991371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:31.110634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:31.112051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57c66c-fac8-47b3-9ac3-3eeea29516e0 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:31.131023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 711/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:33.353204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:33.354099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:33.354099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:33.356081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:33.363051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692f69-7a84-4126-908c-180bd5097a7c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:33.363503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692f69-7a84-4126-908c-180bd5097a7c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:33.363611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692f69-7a84-4126-908c-180bd5097a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:33.440601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-85692f69-7a84-4126-908c-180bd5097a7c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:33.441846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-85692f69-7a84-4126-908c-180bd5097a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:33.456460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:33.556560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:33.595960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:33.598354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:33.719817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59d04-26c9-422f-a6a4-509083f8ace2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:33.733289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:33.800250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:33.800872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:33.801146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:33.801409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:33.803065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:33.918869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:33.920280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d8b1-0302-45bf-8798-17fcda8b95b2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:33.938391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 734/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:36.240123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:36.240706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:36.240706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:36.242941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:36.250935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536e62-80d6-490c-b559-9fd8aabca989 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:36.251386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536e62-80d6-490c-b559-9fd8aabca989 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:36.251481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536e62-80d6-490c-b559-9fd8aabca989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:36.303453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:36.322677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bf536e62-80d6-490c-b559-9fd8aabca989 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:36.323935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bf536e62-80d6-490c-b559-9fd8aabca989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:36.427840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:36.477076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:36.478675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:36.600534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e556e-78d7-4192-b819-b4cc25b42292 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:36.643926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:36.678925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:36.679551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:36.679829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:36.680095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:36.681759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:36.808828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:36.810302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0036b95-f81b-4cb3-8ed6-400949580a79 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:36.838277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 714/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:36:38.962667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:38.963275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:38.963275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:38.964570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:38.970711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67722f2-90c2-4910-a01b-23c31dec34a7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:38.971191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67722f2-90c2-4910-a01b-23c31dec34a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:38.971289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67722f2-90c2-4910-a01b-23c31dec34a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:39.041801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c67722f2-90c2-4910-a01b-23c31dec34a7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:39.043006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c67722f2-90c2-4910-a01b-23c31dec34a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:39.063989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:39.157299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:39.199286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:39.200812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:39.312910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fb3c1-6488-4854-a7ef-bc981e649528 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:39.334573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:39.398225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:39.398953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:39.399241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:39.399518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:39.401168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:39.536373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:39.537852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1abc4-1ed1-45ea-8c0a-9e7f40b2e3f3 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:39.556596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 737/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:41.617727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:41.618338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:41.618338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:41.619652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:41.626385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f8c3-dbfa-4306-bcd0-ca8eb18bef2c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:41.627166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f8c3-dbfa-4306-bcd0-ca8eb18bef2c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:41.627359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f8c3-dbfa-4306-bcd0-ca8eb18bef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:41.697134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:41.699978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bdf5f8c3-dbfa-4306-bcd0-ca8eb18bef2c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:41.701703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bdf5f8c3-dbfa-4306-bcd0-ca8eb18bef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:41.803506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:41.850919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:41.852460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:41.975176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eea936-c641-47ce-99db-99f3421d1f85 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:42.036404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:42.062648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:42.063306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:42.063585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:42.063856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:42.065489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:42.179368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:42.180906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1926b-3c58-4b2f-a0c7-8841059a3735 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:42.201052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 717/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:44.269882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:44.270445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:44.270445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:44.272194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:44.277682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2233a-d80a-4263-a4c7-5e07eff4476d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:44.278161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2233a-d80a-4263-a4c7-5e07eff4476d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:44.278268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2233a-d80a-4263-a4c7-5e07eff4476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:44.351286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fbb2233a-d80a-4263-a4c7-5e07eff4476d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:44.352503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fbb2233a-d80a-4263-a4c7-5e07eff4476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:44.384974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:44.457195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:44.496195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:44.497845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:44.606980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66195084-b919-433a-a2e5-60ebc2fd0871 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:44.636152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:44.683834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:44.684450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:44.684720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:44.684994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:44.686685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:44.804435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:44.805898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf49053-04bb-4539-bf0c-f8e971c97d15 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:44.825134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 740/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:46.895679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:46.896289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:46.896289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:46.897614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:46.904309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec918e-bfa1-4501-81ea-00ecb2542a59 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:46.905094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec918e-bfa1-4501-81ea-00ecb2542a59 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:46.905301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec918e-bfa1-4501-81ea-00ecb2542a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:46.980914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-abec918e-bfa1-4501-81ea-00ecb2542a59 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:46.982252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abec918e-bfa1-4501-81ea-00ecb2542a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:46.990755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:47.088007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:47.128555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:47.130135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:47.247741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45c6b-1c7a-4d7f-b170-64f48829f96d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:47.307735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:47.328824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:47.329452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:47.329757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:47.330039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:47.331657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:47.458824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:47.460550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe5431-5884-4443-95bf-e80343e865ee service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:47.482949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 720/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:36:49.522287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:49.523283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:49.523283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:49.525308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:49.532256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc30ce4-16d6-4c70-866b-b848803019ff None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:49.532811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc30ce4-16d6-4c70-866b-b848803019ff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:49.532811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc30ce4-16d6-4c70-866b-b848803019ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:49.604417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-efc30ce4-16d6-4c70-866b-b848803019ff None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:49.606322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-efc30ce4-16d6-4c70-866b-b848803019ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:49.661940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:49.729975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:49.768694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:49.770269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:49.890560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5ddd-c0b4-4d99-b82f-a3b674eb9919 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:49.935364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:49.982225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:49.982771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:49.983102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:49.983408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:49.985078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:50.114329 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:50.115792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f8e2-78a4-4b2f-9eb5-1dc43717dd51 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:50.134895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 743/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:52.190804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:52.191400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:52.191400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:52.192729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:52.199131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb170-6de4-412e-a1b1-0a497a8efab3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:52.199836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb170-6de4-412e-a1b1-0a497a8efab3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:52.200036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb170-6de4-412e-a1b1-0a497a8efab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:52.272284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c98cb170-6de4-412e-a1b1-0a497a8efab3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:52.273412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c98cb170-6de4-412e-a1b1-0a497a8efab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:52.291353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:52.390022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:52.426657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:52.428185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:52.564050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7f22-88ad-4e1b-93ed-86e6ba6dd17c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:52.636572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:52.644125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:52.644753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:52.645029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:52.645297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:52.647227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:52.770850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:52.772275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808ed0-a67f-4928-b3a3-0494fe75a519 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:52.795518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 723/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:36:54.813094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:54.813701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:54.813701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:54.815034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:36:54.821529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3876-acb6-460f-acb2-52c7647c1d63 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:54.822313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3876-acb6-460f-acb2-52c7647c1d63 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:54.822485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3876-acb6-460f-acb2-52c7647c1d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:54.902246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cc0a3876-acb6-460f-acb2-52c7647c1d63 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:36:54.903486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cc0a3876-acb6-460f-acb2-52c7647c1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:54.956935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:55.012494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:36:55.050366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:36:55.051894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:55.162126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91f9c6-71eb-4d87-8af7-9abca9a3bdaa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:36:55.212967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:36:55.239359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:36:55.239952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:36:55.240225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:36:55.240489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:36:55.242107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:36:55.357483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:36:55.359010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a654e-7d8e-4d2c-9d48-3f7e09e3ce1a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:36:55.379438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 746/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:37:05.010307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=22 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:05.376839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=66 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:14.870634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49ca61-fc2f-4d68-9a04-a75650e57f84 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:14.871539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49ca61-fc2f-4d68-9a04-a75650e57f84 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:14.871539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49ca61-fc2f-4d68-9a04-a75650e57f84 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:14.873730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 725/1471] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:14.880706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c61-4e28-4d0f-82c0-33763f53b476 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:14.881223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c61-4e28-4d0f-82c0-33763f53b476 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:14.881331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c61-4e28-4d0f-82c0-33763f53b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:14.956770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-be849c61-4e28-4d0f-82c0-33763f53b476 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:14.958002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be849c61-4e28-4d0f-82c0-33763f53b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:15.060426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 747/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:15.155153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:15.155441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:15.155719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:15.157158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 726/1473] 10.222.0.47 () {54 vars in 966 bytes} [Mon Aug 1 22:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:15.162426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d94-ceaa-415a-8558-4008f6960f12 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:15.162724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d94-ceaa-415a-8558-4008f6960f12 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:15.162994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d94-ceaa-415a-8558-4008f6960f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:15.243313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-17935d94-ceaa-415a-8558-4008f6960f12 None None] MFA Rules not processed for user `0f05345f84a448f3842f4e91d9f0ac67`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:15.244518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-17935d94-ceaa-415a-8558-4008f6960f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:15.267207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:15.325695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 748/1474] 10.222.0.47 () {58 vars in 1063 bytes} [Mon Aug 1 22:37:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:15.334346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:15.336796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:15.461680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de767f8-1288-4c28-b031-d461728ed754 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:15.536501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:15.537117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:15.537401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:15.537695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:15.539672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:15.595696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:15.669708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:15.671152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a2a4-b947-4065-9c95-fab137d951bf service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:15.691304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 727/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:37:17.774649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:17.775198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:17.775198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:17.777309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 749/1476] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:17.782814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685de9c-512e-42b5-9b20-90aea5add308 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:17.783275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685de9c-512e-42b5-9b20-90aea5add308 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:17.783373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685de9c-512e-42b5-9b20-90aea5add308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:17.851402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1685de9c-512e-42b5-9b20-90aea5add308 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:17.852588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1685de9c-512e-42b5-9b20-90aea5add308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:17.950348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 728/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:17.974548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:17.977035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:18.128930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922652bc-2403-40f9-8537-9ac5a32413bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:18.179665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:18.226295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:18.226916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:18.227187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:18.227459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:18.228155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:18.229957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:37:18.229957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:37:18.229957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:37:18.231087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41640e4d-1cd7-4cc4-aebe-b525b8a05e05 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:18.249883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 750/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Mon Aug 1 22:37:17 2022] GET /identity/v3/projects => generated 2553 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:37:20.258980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:20.259580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:20.259580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:20.260890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 729/1479] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:20.267600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591ef3-2c55-448b-b2da-dad792b70473 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:20.268318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591ef3-2c55-448b-b2da-dad792b70473 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:20.268503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591ef3-2c55-448b-b2da-dad792b70473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:20.343267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-25591ef3-2c55-448b-b2da-dad792b70473 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:20.344565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25591ef3-2c55-448b-b2da-dad792b70473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:20.453117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 751/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:20.508394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:20.510045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:20.520264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:20.662851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b89e-7a7b-42de-bd03-9cbc4cf6af6b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:20.744027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:20.777260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:20.777844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:20.778122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:20.778389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:20.779983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:20.873299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:20.875027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0e1e8-a050-4b54-9f26-9ea039679871 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:20.893717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 730/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:37:23.076219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:23.076791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:23.076791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:23.078253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 752/1482] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:23.084662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed88ef3-9192-44fb-97b0-1d6b25e27ca7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:23.085399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed88ef3-9192-44fb-97b0-1d6b25e27ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:23.085642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed88ef3-9192-44fb-97b0-1d6b25e27ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:23.162168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ed88ef3-9192-44fb-97b0-1d6b25e27ca7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:23.163357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ed88ef3-9192-44fb-97b0-1d6b25e27ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:23.211055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:23.265756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 731/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:23.304107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:23.305637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:23.331359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:23.418593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f133-6950-4522-8cf2-545ab0ed1794 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:23.500453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:23.501089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:23.501365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:23.501665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:23.503289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:23.583242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:23.624973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:23.626461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c0988-8245-4cb2-a4d3-1d21fe13fe3d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:23.646027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 753/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:37:25.706441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:25.707324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:25.707324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:25.709484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 732/1485] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:25.718620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bdc0-71b5-4132-bf3b-7e9ebe128604 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:25.718996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bdc0-71b5-4132-bf3b-7e9ebe128604 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:25.719114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bdc0-71b5-4132-bf3b-7e9ebe128604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:25.797919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d711bdc0-71b5-4132-bf3b-7e9ebe128604 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:25.799041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d711bdc0-71b5-4132-bf3b-7e9ebe128604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:25.924292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 754/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:25.965962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:25.967494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:26.075838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75b506-0561-4396-af8a-f85e2792127d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:26.159206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:26.161798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:26.162508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:26.162866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:26.163228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:26.166356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:26.286774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:26.288218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d4c2-7458-45f9-94a4-465c6801f304 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:26.307371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 733/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:37:28.394420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:28.395025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:28.395025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:28.396242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 755/1488] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:28.402568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947ad4f-8146-4e36-9423-38dc3934a7f0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:28.403037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947ad4f-8146-4e36-9423-38dc3934a7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:28.403143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947ad4f-8146-4e36-9423-38dc3934a7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:28.472253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4947ad4f-8146-4e36-9423-38dc3934a7f0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:28.473474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4947ad4f-8146-4e36-9423-38dc3934a7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:28.531106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:28.575861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 734/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:28.624678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:28.626632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:28.663781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:28.738865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42afe-a876-47f4-8a85-530c9cca9029 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:28.820174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:28.820795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:28.821059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:28.821323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:28.822961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:28.908934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:28.938608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:28.940039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595bc02-b627-48cb-9793-10c399867f65 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:28.958840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 756/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:37:31.160712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:31.161619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:31.161619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:31.163739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 735/1491] 10.222.0.47 () {54 vars in 926 bytes} [Mon Aug 1 22:37:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:37:31.172523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3ea8-fe76-446d-9c8b-815c9815bd5c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:31.172977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3ea8-fe76-446d-9c8b-815c9815bd5c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:31.173083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3ea8-fe76-446d-9c8b-815c9815bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:31.250657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ccaa3ea8-fe76-446d-9c8b-815c9815bd5c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:37:31.251889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ccaa3ea8-fe76-446d-9c8b-815c9815bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:31.356218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 757/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Mon Aug 1 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:37:31.395477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:37:31.397039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:31.511340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cf82-fd9e-4b56-bd41-7e25d82a7766 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:37:31.592600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:37:31.593216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:37:31.593492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:37:31.593796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:37:31.595500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:37:31.604739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:31.716468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:37:31.717937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29698-8583-4e22-b103-2e0e8091f72a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:37:31.737650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 736/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:37:41.354426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=82 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:37:41.735499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=50 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:20.308934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:20.309528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:20.309528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:20.394185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:39:20.395396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:20.501038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 758/1494] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:39:20.510290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:20.510814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:20.510814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:20.598056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:39:20.599178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:20.714883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 737/1495] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:39:20.720263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:20.722076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:20.741514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727eb11-c390-4a94-897b-f80c9428066d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:20.815342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:20.857534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:20.858109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:20.858383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:20.858649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:20.859335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:39:20.859643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:20.860638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:20.860638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:39:20.860638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:39:20.861546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:20.873152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 759/1496] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:39:20.881967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:20.891654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e571e-ba23-443c-a93b-de19bafd2875 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:20.957117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:20.993511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:20.994161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:20.994434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:20.994698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:20.995359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:20.997096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:20.997096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:39:20.997096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:39:20.998324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:21.062321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 738/1497] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:39:20 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:39:21.066714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:21.068246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:21.103948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fbe68-1248-4f5e-b94c-40060e606a08 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:21.214805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:21.215443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.215709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.215973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.216655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:21.217709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:21.217709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:39:21.217709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:39:21.218924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:21.270471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 760/1498] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:39:21 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:39:21.274962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:21.276505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:21.337514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52935a1-9f92-4a49-8101-a82296bd668d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:21.422477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:21.434363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:21.434993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.435270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.435540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.436271 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:21.437304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:21.437304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:39:21.437304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:39:21.438242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:21.451039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 739/1499] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:39:21.455424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:21.456950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:21.469174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376160d-3908-4365-ae9e-fabecca1e37d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:21.495954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:21.571642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:21.572269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.572542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.572784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.573495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:21.574543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:21.574543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:39:21.574543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:39:21.575464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac609b-d643-4983-b5c1-c3fbbb83924c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:21.588742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 761/1500] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:39:21.593109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:21.594766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:21.628531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80685bc4-cc48-4f7e-bb42-236c57121069 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:21.732343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:21.732947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.733228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.733499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] PATH_INFO: `/v3/projects/e64ade119e6540ada4fa21efa08bcd9b/users/91227c31c3964032acf0db45782fa139/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.756407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64ade119e6540ada4fa21efa08bcd9b, user_id=91227c31c3964032acf0db45782fa139, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:21.757440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:21.757440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:39:21.757440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:39:21.759404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa23a7-61a3-4e85-9de1-13ee52760173 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:21.789095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 740/1501] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:39:21 2022] PUT /identity/v3/projects/e64ade119e6540ada4fa21efa08bcd9b/users/91227c31c3964032acf0db45782fa139/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:39:21.801753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfffe1-bb1e-40ee-9fb8-d52fd2636743 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.802067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfffe1-bb1e-40ee-9fb8-d52fd2636743 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.802344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfffe1-bb1e-40ee-9fb8-d52fd2636743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.861073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-54cfffe1-bb1e-40ee-9fb8-d52fd2636743 None None] MFA Rules not processed for user `91227c31c3964032acf0db45782fa139`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:39:21.862315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-54cfffe1-bb1e-40ee-9fb8-d52fd2636743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:21.946730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 762/1502] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:39:21.960346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.960681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.960951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:21.962351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 741/1503] 10.222.0.47 () {54 vars in 964 bytes} [Mon Aug 1 22:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:39:21.967750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc5b3-9374-4279-931a-ac6620b5df55 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:21.968036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc5b3-9374-4279-931a-ac6620b5df55 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:21.968305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc5b3-9374-4279-931a-ac6620b5df55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:22.000404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:22.047150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8a9bc5b3-9374-4279-931a-ac6620b5df55 None None] MFA Rules not processed for user `84a57435ddd44fe79517001c2d91eec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:39:22.048444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8a9bc5b3-9374-4279-931a-ac6620b5df55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:22.130222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 763/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Mon Aug 1 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:39:22.142339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:22.143852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:22.197063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:22.256727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977871c-cd2c-4b10-bed9-5b0d52f19305 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:22.333364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:22.334026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:22.334299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:22.334563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:22.336189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:22.439738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:22.441172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:22.444767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:22.461539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 742/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:39:22.517364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc226d-19ac-4364-a41f-2468dc2001f4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:22.517928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc226d-19ac-4364-a41f-2468dc2001f4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:22.517928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc226d-19ac-4364-a41f-2468dc2001f4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:22.519210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 764/1506] 10.222.0.47 () {52 vars in 830 bytes} [Mon Aug 1 22:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:39:22.523652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:22.525531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:22.634232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6b627-48de-44ea-a776-ab7ee0be34d8 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:22.739150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:22.739784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:22.740062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:22.740334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] PATH_INFO: `/v3/users/91227c31c3964032acf0db45782fa139` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:22.756048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91227c31c3964032acf0db45782fa139)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:22.757650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:22.757650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:39:22.757650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:39:22.759606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:22.772416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:22.833649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91227c31c3964032acf0db45782fa139'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:22.834074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91227c31c3964032acf0db45782fa139'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:22.850389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91227c31c3964032acf0db45782fa139'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:22.860646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91227c31c3964032acf0db45782fa139'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:22.861146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91227c31c3964032acf0db45782fa139'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:22.889098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 743/1507] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:39:22 2022] DELETE /identity/v3/users/91227c31c3964032acf0db45782fa139 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:39:22.894110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:22.894396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:22.894670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:22.952305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:39:22.953506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:23.046313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:23.060033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 765/1508] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:39:23.070569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:23.071808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:23.178570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cca3c-22c1-441c-82b2-0c0ade8cddf9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:23.236506 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:23.257546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:23.258228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:23.258509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:23.258781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:23.260416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:23.376684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:23.378105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe207915-c165-4d41-bde6-10331c91975e service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:23.396821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 744/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:39:23.788003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:39:23.789247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:39:23.803087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9147b-65ad-48a3-b60d-3aaeae95291b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:39:23.913732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:39:23.914484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:39:23.914857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:39:23.915208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] PATH_INFO: `/v3/projects/e64ade119e6540ada4fa21efa08bcd9b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:39:23.931535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64ade119e6540ada4fa21efa08bcd9b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:39:23.933061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:39:23.933061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:39:23.933061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:39:23.934464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:39:24.000280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64ade119e6540ada4fa21efa08bcd9b', 'request_id': 'req-d019eb87-939e-49c9-98db-db4bd5cc7222'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:24.011536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64ade119e6540ada4fa21efa08bcd9b', 'request_id': 'req-d019eb87-939e-49c9-98db-db4bd5cc7222'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:24.019380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64ade119e6540ada4fa21efa08bcd9b', 'request_id': 'req-d019eb87-939e-49c9-98db-db4bd5cc7222'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:24.049539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] The token cache is being invalidate because project e64ade119e6540ada4fa21efa08bcd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:39:24.049839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:24.050354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d019eb87-939e-49c9-98db-db4bd5cc7222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:39:24.051683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 766/1510] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:39:23 2022] DELETE /identity/v3/projects/e64ade119e6540ada4fa21efa08bcd9b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:39:33.394558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=62 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:39:34.047696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:41:06.551976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:06.552882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:06.552882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:06.553958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:06.560329 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f505-0d9e-4aeb-b3da-5d1d51d6488b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:06.560924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f505-0d9e-4aeb-b3da-5d1d51d6488b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:06.561374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f505-0d9e-4aeb-b3da-5d1d51d6488b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:06.678593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7f63f505-0d9e-4aeb-b3da-5d1d51d6488b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:06.679852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7f63f505-0d9e-4aeb-b3da-5d1d51d6488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:06.807421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:07.059615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:07.063505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:07.226619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865c136-3d51-4e46-9568-3fd41c24621a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:07.235434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:07.236446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:07.236874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:07.237329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:07.251443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:07.253334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fea0f-c579-4f22-b893-734efe21f8c7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:07.267255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 746/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:08.230137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efe0-35e5-4090-86f8-5c3f4ede7209 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:08.230742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efe0-35e5-4090-86f8-5c3f4ede7209 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:08.231317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353efe0-35e5-4090-86f8-5c3f4ede7209 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:08.234081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:41:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:08.241463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb7085-eb01-486e-8c0f-64a252cfec52 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:08.242363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb7085-eb01-486e-8c0f-64a252cfec52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:08.242363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb7085-eb01-486e-8c0f-64a252cfec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:08.269822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-67eb7085-eb01-486e-8c0f-64a252cfec52 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:08.270949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-67eb7085-eb01-486e-8c0f-64a252cfec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:08.279146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:08.452385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353efe0-35e5-4090-86f8-5c3f4ede7209 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:08.469010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353efe0-35e5-4090-86f8-5c3f4ede7209 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:08.475166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:08.475810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:08.476088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:08.476362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:08.485851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:08.487486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00dcb3-dab8-4ed8-add6-96035b27fc98 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:08.498340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 769/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:08.792542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c540-2756-423f-b92e-a6f6180aea01 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:08.792762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c540-2756-423f-b92e-a6f6180aea01 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:08.792990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c540-2756-423f-b92e-a6f6180aea01 None None] PATH_INFO: `/v3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:08.794369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 748/1517] 10.222.0.57 () {54 vars in 925 bytes} [Mon Aug 1 22:41:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:41:08.803124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63cb73-dc9a-4e2d-a450-d3f36766544a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:08.803390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63cb73-dc9a-4e2d-a450-d3f36766544a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:08.803668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63cb73-dc9a-4e2d-a450-d3f36766544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:08.872092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5f63cb73-dc9a-4e2d-a450-d3f36766544a None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:08.874252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5f63cb73-dc9a-4e2d-a450-d3f36766544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:08.894085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:41:08.954110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 770/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 1 22:41:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:12.924043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79a8e-6166-4682-a846-98387d43a99b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:12.924579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79a8e-6166-4682-a846-98387d43a99b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:12.924579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79a8e-6166-4682-a846-98387d43a99b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:12.926036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 749/1519] 10.222.0.47 () {54 vars in 961 bytes} [Mon Aug 1 22:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:12.932082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90595331-e56c-4570-b08c-7ab2e2896fba None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:12.932559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90595331-e56c-4570-b08c-7ab2e2896fba None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:12.932667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90595331-e56c-4570-b08c-7ab2e2896fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:12.955260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-90595331-e56c-4570-b08c-7ab2e2896fba None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:12.956512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-90595331-e56c-4570-b08c-7ab2e2896fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:12.966275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 771/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Aug 1 22:41:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:12.974103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb79a8e-6166-4682-a846-98387d43a99b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:12.988922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb79a8e-6166-4682-a846-98387d43a99b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:12.995765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:12.996410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:12.996699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:12.996971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:13.011483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:13.013162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8737a-1388-4cd6-8f46-161bb333fe4a service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:13.024413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 750/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:19.758207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edc92e-7a44-47bd-acdc-8955d51acec5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:19.759117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edc92e-7a44-47bd-acdc-8955d51acec5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:19.759411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edc92e-7a44-47bd-acdc-8955d51acec5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:19.762127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:41:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:19.768652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f956-1ea4-4c9e-8674-1d1d85151b1b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:19.769313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f956-1ea4-4c9e-8674-1d1d85151b1b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:19.769438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f956-1ea4-4c9e-8674-1d1d85151b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:19.799681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f1e9f956-1ea4-4c9e-8674-1d1d85151b1b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:19.800883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f1e9f956-1ea4-4c9e-8674-1d1d85151b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:19.810320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:19.967063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edc92e-7a44-47bd-acdc-8955d51acec5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:19.982578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edc92e-7a44-47bd-acdc-8955d51acec5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:19.988557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:19.989176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:19.989448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:19.989747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:19.999129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:20.000755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35664ca-2cbc-4adf-ba27-8e306b243a6d service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:20.011184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 773/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:20.824157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f68d5-b775-4812-b22c-b9ec4221f7d4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:20.824746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f68d5-b775-4812-b22c-b9ec4221f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:20.824746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f68d5-b775-4812-b22c-b9ec4221f7d4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:20.826092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 22:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:20.831173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6121d-f719-4484-9f66-4d17d1323c54 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:20.831623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6121d-f719-4484-9f66-4d17d1323c54 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:20.831730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6121d-f719-4484-9f66-4d17d1323c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:20.859411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dfd6121d-f719-4484-9f66-4d17d1323c54 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:20.861401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dfd6121d-f719-4484-9f66-4d17d1323c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:20.875704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 22:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:41:21.028413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f68d5-b775-4812-b22c-b9ec4221f7d4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:21.044020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f68d5-b775-4812-b22c-b9ec4221f7d4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:21.049915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:21.050878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:21.051159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:21.051431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:21.060350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:21.062115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071df68-e303-4ce8-8b06-010029ef8d72 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:21.072008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 753/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:24.793484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4966-0306-40cd-af54-98797827040a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:24.794410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4966-0306-40cd-af54-98797827040a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:24.794410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4966-0306-40cd-af54-98797827040a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:24.796621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:41:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:24.803494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149a66-1240-49d1-8dbf-527730e7aee8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:24.804214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149a66-1240-49d1-8dbf-527730e7aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:24.804214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149a66-1240-49d1-8dbf-527730e7aee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:24.832340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d149a66-1240-49d1-8dbf-527730e7aee8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:24.833625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d149a66-1240-49d1-8dbf-527730e7aee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:24.844540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:24.936429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4966-0306-40cd-af54-98797827040a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:24.939789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f96f4966-0306-40cd-af54-98797827040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:25.033330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4966-0306-40cd-af54-98797827040a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:25.037249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:25.037903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:25.038183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:25.038451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:25.047682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:25.049041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc599ad-1fa2-4b2c-830e-872fab944838 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:25.058406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 776/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:27.035910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b0c9-82db-4cdd-801b-916e3f23d44b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:27.036430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b0c9-82db-4cdd-801b-916e3f23d44b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:27.036548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b0c9-82db-4cdd-801b-916e3f23d44b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:27.037911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:27.043593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403099d-dbc2-4a73-bbf7-4cf6a3fb9061 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:27.044100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403099d-dbc2-4a73-bbf7-4cf6a3fb9061 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:27.044100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403099d-dbc2-4a73-bbf7-4cf6a3fb9061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:27.064893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0403099d-dbc2-4a73-bbf7-4cf6a3fb9061 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:27.066130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0403099d-dbc2-4a73-bbf7-4cf6a3fb9061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:27.074329 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:41:27.111568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9b0c9-82db-4cdd-801b-916e3f23d44b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:27.124788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9b0c9-82db-4cdd-801b-916e3f23d44b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:27.128967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:27.129599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:27.129881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:27.130150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:27.145430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:27.146929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4418db0-4131-415d-82ae-57dda28cdd1d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:27.157424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 756/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:30.643402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:30.644038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:30.644038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:30.645789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:41:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:30.653878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c63f-b636-4f88-a6df-34c17822d095 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:30.654668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c63f-b636-4f88-a6df-34c17822d095 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:30.654840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c63f-b636-4f88-a6df-34c17822d095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:30.681674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-90f4c63f-b636-4f88-a6df-34c17822d095 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:30.682858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-90f4c63f-b636-4f88-a6df-34c17822d095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:30.692251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:31.002893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:31.005908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:31.086068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:41:31.101521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f2c2-2e78-4079-a0d1-87f95d207f4c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:31.105273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:31.105933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:31.106208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:31.106481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:31.115785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:31.117254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061108fc-9dc6-4f81-b306-349abe192829 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:31.126806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 779/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:33.152252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:33.153171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:33.153171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:33.154981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 22:41:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:41:33.164400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46758a49-2c4d-44fd-91d6-7a26b161a8ea None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:33.164893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46758a49-2c4d-44fd-91d6-7a26b161a8ea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:33.165377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46758a49-2c4d-44fd-91d6-7a26b161a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:33.189647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-46758a49-2c4d-44fd-91d6-7a26b161a8ea None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:41:33.190861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-46758a49-2c4d-44fd-91d6-7a26b161a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:33.202411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 22:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:41:33.509488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:41:33.512650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:41:33.522446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:41:33.530056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb81e2-3d2f-4d66-9d74-268d74bce839 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:41:33.536561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:41:33.537432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:41:33.537832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:41:33.538197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:41:33.551785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:41:33.553252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789369-4e9e-4c4f-899a-384ad42188c3 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:41:33.563331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 759/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:41:43.184199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=11 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:41:43.559010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:03.388429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.392746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.392746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.415177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.416526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.428304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 781/1540] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.437884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d68fc-aa6f-42ce-8e1a-8f9e0c167762 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.438509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d68fc-aa6f-42ce-8e1a-8f9e0c167762 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.438509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d68fc-aa6f-42ce-8e1a-8f9e0c167762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.466399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ba4d68fc-aa6f-42ce-8e1a-8f9e0c167762 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.467828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ba4d68fc-aa6f-42ce-8e1a-8f9e0c167762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.478348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 760/1541] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.483558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:03.499789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a801ecd-d0ee-45ce-947a-035b13cf92a5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:03.504730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:03.505539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.505849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.506124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.506841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:03.507144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:03.508771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:03.527595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 782/1542] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:03.530497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7762ef-af22-4646-8037-3bc45c91d455 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.530787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7762ef-af22-4646-8037-3bc45c91d455 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.531060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7762ef-af22-4646-8037-3bc45c91d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.536150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:03.563195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c23b-868a-4078-a42a-edeab5258381 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:03.569465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6e7762ef-af22-4646-8037-3bc45c91d455 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.570711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6e7762ef-af22-4646-8037-3bc45c91d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.576061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:03.576699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.576972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.577237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.578128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:03.580087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:03.580087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:43:03.580087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:43:03.581309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f702-6e0f-4b9b-883d-761ff1047bcd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:03.593147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 761/1543] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.601619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.601911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.602190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.646043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.649165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 783/1544] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:03.650928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258896-2d48-4310-8e0b-97448f6d98d3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.651246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258896-2d48-4310-8e0b-97448f6d98d3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.651557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258896-2d48-4310-8e0b-97448f6d98d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.658244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.706787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ed258896-2d48-4310-8e0b-97448f6d98d3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.708358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ed258896-2d48-4310-8e0b-97448f6d98d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.766184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 762/1545] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.767406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:03.793439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3705-54f3-4050-a8df-22030e0e7a18 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:03.803473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:03.804537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.806357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.806808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.808515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:03.810744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:03.810744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:43:03.810744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:43:03.812553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51967110-ff42-4a60-b436-75864c0e616a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:03.838513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 784/1546] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.841641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f00-cdf1-443e-bed1-0e7bf7ee02fb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.842206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f00-cdf1-443e-bed1-0e7bf7ee02fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.842642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1f00-cdf1-443e-bed1-0e7bf7ee02fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.862282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 763/1547] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:03.863901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.864353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.864791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.884840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-970a1f00-cdf1-443e-bed1-0e7bf7ee02fb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.886104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-970a1f00-cdf1-443e-bed1-0e7bf7ee02fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.892433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.893689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.895456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 785/1548] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.901544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.901544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.901544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.910028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 764/1549] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.911105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:03.931638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.931638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.935333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839eb2-db74-409e-97ff-4f6fc45e8e28 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:03.938443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 786/1550] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.939264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:03.943141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:03.944003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.944444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.944973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.945875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:03.946461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:03.947498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:03.947498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:43:03.947498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:43:03.948433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e253d-aab4-47a8-bb85-5cca6c96c396 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:03.951458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901e39c-e7a3-4ae2-8491-ef9e50f2e41c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:03.956327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:03.957077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.957355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.957655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.958493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:03.960088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574cf96-c7cd-48e5-849a-cc9b7262179d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:03.960776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 765/1551] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:03.962111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa303-e0e6-44a0-b482-ecd34f7fdd3e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.962661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa303-e0e6-44a0-b482-ecd34f7fdd3e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.962941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa303-e0e6-44a0-b482-ecd34f7fdd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.971189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 787/1552] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:03.972420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44e6a-4223-4cc7-83f1-c4c56090ed76 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.972708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44e6a-4223-4cc7-83f1-c4c56090ed76 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.972978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44e6a-4223-4cc7-83f1-c4c56090ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:03.982956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8f2aa303-e0e6-44a0-b482-ecd34f7fdd3e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.984154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f2aa303-e0e6-44a0-b482-ecd34f7fdd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.992253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-87f44e6a-4223-4cc7-83f1-c4c56090ed76 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:03.992390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 766/1553] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:03.993544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-87f44e6a-4223-4cc7-83f1-c4c56090ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:03.994903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:03.995377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:03.995849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.005196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 788/1554] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.006692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797556-e6c5-4d65-9118-bd8eee9edb73 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.006976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797556-e6c5-4d65-9118-bd8eee9edb73 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.007242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797556-e6c5-4d65-9118-bd8eee9edb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.020146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.021737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.027889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8a797556-e6c5-4d65-9118-bd8eee9edb73 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.029049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8a797556-e6c5-4d65-9118-bd8eee9edb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.030155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 767/1555] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.030937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.039224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 789/1556] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.040504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.040813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.041317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.041550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd953-42a3-457b-95c9-ae5cfa1a19df None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.047297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.047922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.048200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.048462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.049232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.049535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.051539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.063981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.065697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.070482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 768/1557] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.071791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.076403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 790/1558] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.077067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.087581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43fde6-fc92-4293-a8d4-853117f6ed1e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.087839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1535d06-9dfe-4087-adef-886cdd5b20e3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.092849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.092984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.093473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.093637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.093833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.093926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.094113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.094222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.094771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.094891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.096396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c2e4e-8386-4790-a21a-d2232c603417 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.096663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.107816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 791/1559] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:04.109060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.109461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.109878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.129720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 769/1560] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.130623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.131603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.133069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.183495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.190060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c16b67-5b73-4524-b455-9bbb6246b0fc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.194561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.195180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.195454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.195756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.196480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.196775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.197209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 792/1561] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.198111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.202378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdfe2-292e-4992-b176-6915f5de9c87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.205577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.207919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca5b9b-25d2-499b-99f1-679d1635feb2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.213499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.214692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.214995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.215292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.216143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.223281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.223462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 770/1562] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.223462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701fe03-45a0-40f0-8211-7bec86881c51 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.223641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.223744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701fe03-45a0-40f0-8211-7bec86881c51 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.223744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701fe03-45a0-40f0-8211-7bec86881c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.236217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 793/1563] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.236217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.241075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65799a-549d-4eb8-8bd0-5fb36ed2d98c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.242941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e701fe03-45a0-40f0-8211-7bec86881c51 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.244163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e701fe03-45a0-40f0-8211-7bec86881c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.245996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.246604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.246863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.247131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.247785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.249704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.252331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 771/1564] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.253516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00862782-2065-40b4-9944-bd421507e652 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.253836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00862782-2065-40b4-9944-bd421507e652 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.254110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00862782-2065-40b4-9944-bd421507e652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.273103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-00862782-2065-40b4-9944-bd421507e652 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.274459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-00862782-2065-40b4-9944-bd421507e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.282488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 794/1565] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.283230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 772/1566] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.283447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.283969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00862782-2065-40b4-9944-bd421507e652 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.298345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00862782-2065-40b4-9944-bd421507e652 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.351140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff4915-c129-4f30-86f6-2a9b7df6b7d2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.353749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.354417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.354767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.355043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.355716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.356096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.356771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.357051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.357325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] PATH_INFO: `/v3/projects/6a6afb01194249d6af1c369e9f109b72/users/a8bbeb0e65a54b1199d0fe718168af70/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.357684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.373538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6afb01194249d6af1c369e9f109b72, user_id=a8bbeb0e65a54b1199d0fe718168af70, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.374716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:04.374716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:43:04.374716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:43:04.376772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.381194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 773/1567] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.382178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.390926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 795/1568] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:04 2022] PUT /identity/v3/projects/6a6afb01194249d6af1c369e9f109b72/users/a8bbeb0e65a54b1199d0fe718168af70/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:04.392478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.392677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df106-cf1d-43b2-b42b-01d60dcce596 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.452656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.453377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.453496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139acafc-8367-4544-a780-fb7a3013e44d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.453652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.453839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.454536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.454838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.456445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.458736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.459364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.459643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.460089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.460752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.464400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.469260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 774/1569] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.470109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.480686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e8428-5b48-4d62-8c4a-ce6195cf0ef3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.486372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.487035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.487321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.487598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.488281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.490394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.500049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 796/1570] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.500049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.522509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 775/1571] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.522509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.531641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.566013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e952102-1eb3-47c1-9798-66d2e106c76a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.568773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.592782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da062f7a-b361-4666-a795-e95a93fcb76d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.597911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.598539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.598816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.599084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.599744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.601696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.627197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 776/1572] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.627197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.627454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.627454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.627655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.627940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.628629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.628959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.630614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.634420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657803-7bf0-41d2-bea8-4ee99b427c02 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.640531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.641152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.641426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.641736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.642423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:04.642720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.643665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 797/1573] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.644384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1b3b7-636c-4629-a4e5-11856a731ff7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.644611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.655222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f93c-141f-41f6-bdd5-f4c5a6a55ee6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.657931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 777/1574] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:04.659156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.659458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.659715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.661137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.661881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.662187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.662543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.663501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.665321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.675953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 798/1575] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:04.679211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.681427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:04.682819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:04.687837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad5653-c630-47b6-9d07-f6e4b91d5a63 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.691091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 778/1576] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:04.691942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.692621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.693212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.693482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.693782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.694581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.696472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.701820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807206-7ddc-48d2-9008-2d0a59ff8f10 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.706465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.707017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.707300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.707575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.708240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.710211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.729815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 799/1577] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.729815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.738660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 779/1578] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/users => generated 278 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.739499 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.749402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177dd02-727a-47b9-8b4d-6dde5cf208b5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.790676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62150c4-c947-4d27-b800-5ddcfcaa2f69 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.795476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.796185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.796510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.796806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.797544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.799663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.809107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.809768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.810042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.810308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.811019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.812735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.822167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 800/1579] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.822975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.829368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 780/1580] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:04.829368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.832338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e70598-2d9c-4660-81cb-f68432ab9636 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.836926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.837531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.837840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.838135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.838519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc997fe-ddc2-4e97-a0f9-4706f56f619d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.839253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.841161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.843490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.844087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.844363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.844634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.845283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.847329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:04.873313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 801/1581] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.874305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.874446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 781/1582] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:04 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:04.875309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:04.881013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.932564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b129af-15da-40ca-9e8c-2612b6442def admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.936570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.937199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.937472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378283f-2a0a-4559-bd30-92dbbd638575 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:04.937600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.937774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.963507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:04.994508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:04.995131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:04.995430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:04.995694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:04.996388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:04.998016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.000744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.002238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.008666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 802/1583] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.010411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.012767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 782/1584] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:05.013627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.023219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92185-33f3-4b7b-b78d-768baa87412c service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.023516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97879c-9093-4254-9a4f-f40ccef95129 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.028034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.028534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.028686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.028924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.029149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.029262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.029427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.029774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.029917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.030438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.031889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.032411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.045604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 783/1585] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.045604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.055170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae6c2-138c-4e7f-8fd2-394b046f3fa0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.062412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 803/1586] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:05.062602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.062726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.062828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.062828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.062828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.062828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.063475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.073689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a639-ff5d-4649-84b4-4212c1646c5f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.074473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 784/1587] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.075129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.082260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.082260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.082558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.082889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.083666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.085380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.085618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a6380-7df5-4804-82d1-ab8a53c0b886 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.090768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.092126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.092417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.092687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.093359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.095321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.099512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 804/1588] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.100455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.112636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f1c89-6427-4c86-9928-1b575584082c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.118000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.118611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.118892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.119169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.119818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.122019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.126622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 785/1589] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:05.129735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.141474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9f5a-e8cf-4e08-99f7-309b7cd7c6b5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.146225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.149698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 805/1590] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:05.149976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.149976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.149976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] PATH_INFO: `/v3/projects/b32a8358a242425db77284978619d416/users/e847171f82374d59bb01616636385a23/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.150272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.161625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372a5cc-1c08-48d3-8998-1a39f189282e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.166589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.167242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32a8358a242425db77284978619d416, user_id=e847171f82374d59bb01616636385a23, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.167386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.167540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.167809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.168739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.170066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.170499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.183045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 806/1591] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.183961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.188588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 786/1592] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/b32a8358a242425db77284978619d416/users/e847171f82374d59bb01616636385a23/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:05.189432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.194656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26f5e-a806-4c33-84f3-eb6573d3857b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.265052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:05.271270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793cc2-06f5-4d13-ac40-8b5be48fd5fa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.276308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.277186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.277464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.277803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.278507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.280138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.280375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.280991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.281303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.281760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.282863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.284454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.290833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 787/1593] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.291658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.298605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 807/1594] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.299463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.305657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00049-dba6-4c42-9d80-15c73f113806 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.310717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511999e-d31e-4cb1-9d3f-f43fecb097e6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.310845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.311480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.311764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.312037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] PATH_INFO: `/v3/projects/dce176fe07434b6bbf6fa0c031c2903f/users/e6d336332a65408c8a438c8f96f4415d/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.316356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.317123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.317752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.317752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.318437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.320181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.328288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce176fe07434b6bbf6fa0c031c2903f, user_id=e6d336332a65408c8a438c8f96f4415d, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.331004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.332896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 808/1595] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.335231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.346588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed996a-ab92-4ef6-9c31-38e31ec11cb1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.347200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 788/1596] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/dce176fe07434b6bbf6fa0c031c2903f/users/e6d336332a65408c8a438c8f96f4415d/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:05.348136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.379733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:05.415490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.416240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.416637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.417122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.417955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.419762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3377f4a-16ff-41ad-835e-627ca8263f77 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.433776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 809/1597] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.433776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.433776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.433776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.438480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd811f52-3860-4afe-81fa-31a47c2994b2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.445667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.446653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.447099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.447542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] PATH_INFO: `/v3/projects/7223873e6eac420abffcf2ea14e602ca/users/fad57ee650d8414eacceee3fa5bb9286/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.466575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] MFA Rules not processed for user `e847171f82374d59bb01616636385a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:05.467581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:05.477930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7223873e6eac420abffcf2ea14e602ca, user_id=fad57ee650d8414eacceee3fa5bb9286, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.482441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.503689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 789/1598] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/7223873e6eac420abffcf2ea14e602ca/users/fad57ee650d8414eacceee3fa5bb9286/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:05.504908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.558186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 810/1599] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:05.558186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.568355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada07d83-c42e-4b18-92ba-3d9102deb2dd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.584474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a85ad-8437-4f9f-9504-5e81c061adb9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.590444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.591218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.591557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.591914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] PATH_INFO: `/v3/projects/41e72072c5cc453fbc31425b050a07c1/users/a2ff5f41f9b44c2c9f5f90f13c266fbd/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.609168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e72072c5cc453fbc31425b050a07c1, user_id=a2ff5f41f9b44c2c9f5f90f13c266fbd, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.612054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.627595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 790/1600] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/41e72072c5cc453fbc31425b050a07c1/users/a2ff5f41f9b44c2c9f5f90f13c266fbd/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:05.628319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.637766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.638409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.638696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.638974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.639718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.641334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa53d1-a754-49ff-8370-1dc3664fd9fd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.643982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:05.653953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 811/1601] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.655170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064701d-1020-4392-9f9e-e3ce3ba22530 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.655458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064701d-1020-4392-9f9e-e3ce3ba22530 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.655731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064701d-1020-4392-9f9e-e3ce3ba22530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.677250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9064701d-1020-4392-9f9e-e3ce3ba22530 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:05.678686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9064701d-1020-4392-9f9e-e3ce3ba22530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:05.713257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c87f9-ca75-402e-9007-25a0b80456eb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.723389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.724239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.724560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.724861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.725605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.728375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.744205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 791/1602] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.745520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.756087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 812/1603] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:05.757234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.757493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e28e-7af0-485c-adca-98bf0d6be3ba admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.757666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.757945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.780844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.781502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.781805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.782079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.782804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.784420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.791488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] MFA Rules not processed for user `fad57ee650d8414eacceee3fa5bb9286`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:05.794274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:05.795837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 792/1604] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:05.796777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.808599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53737688-1fcd-4764-a119-a05fcc2cba53 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.815006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.816257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.816685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.817144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.817990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:05.820289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b34b-0947-4f3a-8a5a-d3e8909c93c2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:05.853353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:05.875311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 793/1605] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:05.877242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.877569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.877856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.893938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 813/1606] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:05.895709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.901985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:05.903218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:05.912892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae258a-d5f1-4a9a-823c-32156cfac67f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:05.981070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 794/1607] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:05.982762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:05.988646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:05.989338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:05.989861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:05.990160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] PATH_INFO: `/v3/projects/fcf1ca7cb1194e0ca37da94225a1e139/users/8f6f6e5693dc4f0aa985b6313b6ffcbe/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:05.994328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7768d-eb36-45b4-90c8-3954017fad0f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.003733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.004864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.005358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.005818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] PATH_INFO: `/v3/projects/96cf343ab1cd4658997298d18a083f29/users/527b7873931742049db9f44104396686/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.016826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf1ca7cb1194e0ca37da94225a1e139, user_id=8f6f6e5693dc4f0aa985b6313b6ffcbe, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.020706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.029297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cf343ab1cd4658997298d18a083f29, user_id=527b7873931742049db9f44104396686, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.032310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.045855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 814/1608] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/fcf1ca7cb1194e0ca37da94225a1e139/users/8f6f6e5693dc4f0aa985b6313b6ffcbe/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:06.045855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.045855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:06.061242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 795/1609] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:05 2022] PUT /identity/v3/projects/96cf343ab1cd4658997298d18a083f29/users/527b7873931742049db9f44104396686/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:06.062215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.070957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:06.130034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9b9ef-d9bd-46f5-aba6-b0860f77128f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.138757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cd1fa-da8c-4b54-9677-754c93ca0353 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.144108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.145040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.145454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.145894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] PATH_INFO: `/v3/projects/6e61cb6b37994196a0ef1b5910524917/users/26104e6d2a214fa69139e2bf90b53c74/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.164045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e61cb6b37994196a0ef1b5910524917, user_id=26104e6d2a214fa69139e2bf90b53c74, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.167455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.188031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 796/1610] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:06 2022] PUT /identity/v3/projects/6e61cb6b37994196a0ef1b5910524917/users/26104e6d2a214fa69139e2bf90b53c74/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:06.189260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.214342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.214716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.214995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.215264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.233063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.234465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.245314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 815/1611] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:06.246243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.256295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f36b3-5958-4428-a9f9-44f8fce66c01 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.289851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:06.295691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202e58-513a-419a-8fda-4a363f9cdefe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.307995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.311187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.311187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.311187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.311187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.313060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.349753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 797/1612] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:06.350684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.355830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.356551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.357161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.357525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.360186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.362852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182da3cd-6b41-4d26-85ae-d6fc07ea1f9d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.407702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 816/1613] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/projects => generated 355 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:06.407702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417660a-5375-4da3-b942-946602e4feda None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.407702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417660a-5375-4da3-b942-946602e4feda None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.408130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417660a-5375-4da3-b942-946602e4feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.422409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d05275-99d7-4b7d-a5ab-2d1af695af26 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.462816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2417660a-5375-4da3-b942-946602e4feda None None] MFA Rules not processed for user `8f6f6e5693dc4f0aa985b6313b6ffcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:06.466830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2417660a-5375-4da3-b942-946602e4feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:06.505226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.506242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.506617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.507083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.543888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:06.570373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 817/1614] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:06.571964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14041a-afe7-4866-9fc1-f08b27650d99 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.572247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14041a-afe7-4866-9fc1-f08b27650d99 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.572522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14041a-afe7-4866-9fc1-f08b27650d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.607579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5b14041a-afe7-4866-9fc1-f08b27650d99 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:06.608921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b14041a-afe7-4866-9fc1-f08b27650d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:06.627925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 818/1615] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:06.629482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eae17-ecbf-4ca9-b702-38d53e17b02c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.629818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eae17-ecbf-4ca9-b702-38d53e17b02c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.630223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eae17-ecbf-4ca9-b702-38d53e17b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.635916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.639307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.654778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 798/1616] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:06.662757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.676321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6aad-8734-4cb0-a22d-85c544bea65b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.676735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e08eae17-ecbf-4ca9-b702-38d53e17b02c None None] MFA Rules not processed for user `26104e6d2a214fa69139e2bf90b53c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:06.677680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e08eae17-ecbf-4ca9-b702-38d53e17b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:06.688401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.689814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.690231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.690695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.691526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.693353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.707572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 799/1617] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:06.712267 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.731215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5a1f-3a8f-41ff-9c06-d2eed5310ff3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.749524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.750493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.750862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.751294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.752177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.755087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.811430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 800/1618] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:06.815619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.834754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 819/1619] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:06.836778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e11dc-ebb3-4c9a-a071-c7abc9df0c17 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.837273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e11dc-ebb3-4c9a-a071-c7abc9df0c17 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.838642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e11dc-ebb3-4c9a-a071-c7abc9df0c17 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.842856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 820/1620] 10.222.0.47 () {54 vars in 962 bytes} [Mon Aug 1 22:43:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:06.843007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d64b40-0923-455b-ae65-22476509ae13 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.848965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.848965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.848965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.848965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.856505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e11dc-ebb3-4c9a-a071-c7abc9df0c17 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.860970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.863009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.875112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e11dc-ebb3-4c9a-a071-c7abc9df0c17 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.878588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 801/1621] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:06.879745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.898130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b7f6-9670-4236-940f-1a7d82b9f06b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.904619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.907514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.907850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.908171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.909738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.910484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.911403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.911707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.912115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.912519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.913473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.915491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83812971-646a-48a8-b50d-2b6d9aebe609 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:06.923319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:06.930344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 821/1622] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:06.931495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:06.936809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 802/1623] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:06.938819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.939463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.939961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.947747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3446e-49d1-449c-9250-dc6fc05ad5b8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:06.955272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:06.955974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:06.956305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:06.956580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:06.957348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:06.960956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.013907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 822/1624] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:07.014756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.015991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:07.019682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:07.104958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867b-385a-4948-9709-ecf5147650a2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.114490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.115207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.115486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.115762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] PATH_INFO: `/v3/projects/f5ef02eba6714a0795d77c55cf868241/users/784f128bbb91461aa0fb7ad098b8284e/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.134817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ef02eba6714a0795d77c55cf868241, user_id=784f128bbb91461aa0fb7ad098b8284e, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.137837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.138707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 803/1625] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Aug 1 22:43:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:07.139772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.154761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe47a-f0c4-4b11-b5dd-c62373e4a93b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.156586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 823/1626] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:07 2022] PUT /identity/v3/projects/f5ef02eba6714a0795d77c55cf868241/users/784f128bbb91461aa0fb7ad098b8284e/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:07.157544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.189003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:07.245401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb50bc-b0fe-4230-b3dc-7abb43a96683 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.251142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.251968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.252254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.252970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.253198 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.255194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.291653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 824/1627] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:07 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:07.292610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.307366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.308775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.309333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.310002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] PATH_INFO: `/v3/projects/ef6d7852e147412784f42d75031ffe59/users/9a11e8c6fb0144919b9d511f72fba6fc/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.332284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6d7852e147412784f42d75031ffe59, user_id=9a11e8c6fb0144919b9d511f72fba6fc, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.335357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493acde-dc98-4292-9fde-f188d4da03ac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.363550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 804/1628] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:07 2022] PUT /identity/v3/projects/ef6d7852e147412784f42d75031ffe59/users/9a11e8c6fb0144919b9d511f72fba6fc/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:07.364789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.365080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.365354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.421931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340bd4c-a52e-4b68-9073-4ee55c360457 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.424736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] MFA Rules not processed for user `784f128bbb91461aa0fb7ad098b8284e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:07.428248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:07.538872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.540717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.541612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.541995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.569854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:07.599475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 805/1629] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:07.600898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.627115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:07.654350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.658231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.671992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 825/1630] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:07.674344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.738158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24a576-773f-40af-b9a6-0633b205452f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.752875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.753776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.755094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.757120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.758843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.765460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.780870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 806/1631] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:07.782505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.800153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98732cf-2eea-41ba-8944-5143a4872bd4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.809600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d993-9d73-4310-a2d4-e5c7c0e3f27f service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.810148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.811223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.811824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.812395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.813331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.815652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.825641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.825641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.825641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.825641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.826405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.826405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:07.858659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 826/1632] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:07.859382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.884705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be69ff-3737-4975-a2d4-8fe0aaa7778f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:07.913827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:07.914966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:07.917757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:07.918179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] PATH_INFO: `/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0/users/31c723d379b449a091eb41bb0226a50f/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:07.927778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 807/1633] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:07 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:07.928799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:07.973306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4032e865e34f8ebdad024fc80fa1c0, user_id=31c723d379b449a091eb41bb0226a50f, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:07.984812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.028382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 827/1634] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:07 2022] PUT /identity/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0/users/31c723d379b449a091eb41bb0226a50f/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:08.029502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.149950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308df93-1e21-48ca-9d98-21ce3fb05b15 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.202197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a99d9-8737-41a5-99ed-fd2b406493a6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.212036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.213365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.214382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.215178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.216235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.228656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.279543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 828/1635] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:08 2022] POST /identity/v3/users => generated 300 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:08.284418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.313628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.317617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171f66-383c-4bc2-a6d0-a7427a8b0377 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.318331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.318743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.318960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] PATH_INFO: `/v3/projects/ef6d7852e147412784f42d75031ffe59/users/9a11e8c6fb0144919b9d511f72fba6fc/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.327152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6d7852e147412784f42d75031ffe59, user_id=9a11e8c6fb0144919b9d511f72fba6fc, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.335007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72c770-a818-47cc-bcdf-0370a9fb1257 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.337126 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.338421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.339056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.339468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.340199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.361486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2aebbf-f459-4b65-8805-b55a89cd19d3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.364997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 808/1636] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:07 2022] PUT /identity/v3/projects/ef6d7852e147412784f42d75031ffe59/users/9a11e8c6fb0144919b9d511f72fba6fc/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:08.376176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4e72e-4c69-476c-bdaf-400665fcabd9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.377278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4e72e-4c69-476c-bdaf-400665fcabd9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.378323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4e72e-4c69-476c-bdaf-400665fcabd9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.380077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 809/1637] 10.222.0.47 () {54 vars in 912 bytes} [Mon Aug 1 22:43:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:08.380747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4e72e-4c69-476c-bdaf-400665fcabd9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.392864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 829/1638] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:08.395027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.396526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:08.398761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.399991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.449836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] MFA Rules not processed for user `9a11e8c6fb0144919b9d511f72fba6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:08.451122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:08.499710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:08.499918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4e72e-4c69-476c-bdaf-400665fcabd9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.518077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.518724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.519009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.519280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.519993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.526711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7968c-fe0c-4d3e-9019-b96664063f99 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.551070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 810/1639] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:08.552561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.552809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.555770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.600415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:08.603893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:08.620939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 830/1640] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:08.622864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.644285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4cd1a-849f-42b8-8b57-37508ceb3ba0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.654742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.657448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.658104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.658708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] PATH_INFO: `/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0/users/31c723d379b449a091eb41bb0226a50f/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.663739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4032e865e34f8ebdad024fc80fa1c0, user_id=31c723d379b449a091eb41bb0226a50f, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.668191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.691964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 831/1641] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:08 2022] PUT /identity/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0/users/31c723d379b449a091eb41bb0226a50f/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:08.694127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.770050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 811/1642] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:08.770959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:08.824963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb05b20-1c75-40f4-8dc3-fe7b176d67bc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.831347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.832865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.833444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.833974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] PATH_INFO: `/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2/users/56b163434a6a4b748e3b86343bcb9794/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.855320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f321-0eb4-4e78-a7c4-f752183db268 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:08.861790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57bcb4f3eb4428b80d64858b6a34ad2, user_id=56b163434a6a4b748e3b86343bcb9794, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:08.868876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80b8d-f93b-43ae-a09e-6f9fed9a3c73 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:08.901743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 832/1643] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:08 2022] PUT /identity/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2/users/56b163434a6a4b748e3b86343bcb9794/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:08.909157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.910679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.911403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.935782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:08.936432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:08.936712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:08.937033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:08.945118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:08.978830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] MFA Rules not processed for user `31c723d379b449a091eb41bb0226a50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:08.981084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:09.069583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.071060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.085298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 833/1644] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:09.086547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:09.097159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 812/1645] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:09.098077 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:09.108460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:09.118807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5bf70-ba71-464e-90b1-a497d45c1588 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:09.223387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2a63-af24-43a9-91d6-41156d90ff19 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:09.228151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:09.229228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.229583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.229861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.237573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:09.238223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.238558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.238814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.239535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.241595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.267348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 813/1646] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:09.267348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:09.287891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186742-3eb0-4c81-af68-62ba1c1338bb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:09.293313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:09.294016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.294287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.294564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.305170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.306791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.320038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 814/1647] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:09.320888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:09.331414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61201-1ebd-44e7-964e-65d7d785832b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:09.338530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:09.339722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.340018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.340287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] PATH_INFO: `/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2/users/56b163434a6a4b748e3b86343bcb9794/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.342444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57bcb4f3eb4428b80d64858b6a34ad2, user_id=56b163434a6a4b748e3b86343bcb9794, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.345310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d10699-0da9-4367-a2d5-f11de3d7b84f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.365605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 815/1648] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:09 2022] PUT /identity/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2/users/56b163434a6a4b748e3b86343bcb9794/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:09.374918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.376213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.377786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.402695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.404339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.429591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 834/1649] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:09.459332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] MFA Rules not processed for user `56b163434a6a4b748e3b86343bcb9794`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:09.461102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:09.629226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:09.639747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d56f3-ccfb-4646-8e04-9ee10e061d2f service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:09.642350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:09.658516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 816/1650] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:09.766674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:09.767391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:09.767671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:09.767943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:09.895439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:09.899102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:09.919925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 835/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:43:10.297812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:10.312617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b507f-d5c0-46d5-a7c8-0f296368fdfe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:10.324745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:10.325438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:10.325770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:10.326046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:10.475741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:10.477935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5248a3e-d8ee-4e88-ab47-862c53ca0f72 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:10.489753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 817/1652] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:10.557490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:10.570315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e97d40-fef6-45f7-9814-16dc7327577c service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:10.582276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:10.582931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:10.586658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:10.588326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:10.613358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:10.628804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:10.630400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400e115-4beb-4336-8c75-4f0d6345a1b7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:10.661202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 836/1653] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:11.533947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f959d9c-a425-43c2-80ad-29b516c3de5f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:11.534464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f959d9c-a425-43c2-80ad-29b516c3de5f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:11.534532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f959d9c-a425-43c2-80ad-29b516c3de5f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:11.535917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 818/1654] 10.222.0.47 () {54 vars in 962 bytes} [Mon Aug 1 22:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:11.546721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd25f4-aaf1-48d8-9af0-d37add448026 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:11.547031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd25f4-aaf1-48d8-9af0-d37add448026 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:11.547270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd25f4-aaf1-48d8-9af0-d37add448026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:11.594969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8edd25f4-aaf1-48d8-9af0-d37add448026 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:11.596776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8edd25f4-aaf1-48d8-9af0-d37add448026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:11.814572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 837/1655] 10.222.0.47 () {58 vars in 1059 bytes} [Mon Aug 1 22:43:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:11.836117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997245-8700-4801-89cc-7a8390412197 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:11.836408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997245-8700-4801-89cc-7a8390412197 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:11.836687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57997245-8700-4801-89cc-7a8390412197 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:11.838098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 819/1656] 10.222.0.47 () {54 vars in 912 bytes} [Mon Aug 1 22:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:11.848197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:11.848714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:11.849152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:11.912007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:11.915145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:11.944465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 838/1657] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:43:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:11.968198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57997245-8700-4801-89cc-7a8390412197 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:11.982439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57997245-8700-4801-89cc-7a8390412197 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:11.987163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:11.988178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:11.988915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:11.989398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:12.021184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:12.023549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b67c0-25da-4cc2-8190-7618fbee0fc1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:12.046878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 820/1658] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:13.177378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:13.220560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520850-e35a-4fb7-976e-905295f1a350 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:13.240725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:13.244748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:13.245318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:13.248457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:13.518689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:13.520204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65158b00-8aa4-4f03-9a13-dff8fcc9d17a service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:13.571176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 839/1659] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:17.232551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:17.233179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:17.233179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:17.301615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] MFA Rules not processed for user `a8bbeb0e65a54b1199d0fe718168af70`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:17.302864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:17.471980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 821/1660] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:17.485393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26c27-747b-4cfd-91e1-08b8e56cf37d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:17.486297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26c27-747b-4cfd-91e1-08b8e56cf37d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:17.486578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d26c27-747b-4cfd-91e1-08b8e56cf37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:17.513251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-72d26c27-747b-4cfd-91e1-08b8e56cf37d None None] MFA Rules not processed for user `a8bbeb0e65a54b1199d0fe718168af70`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:17.514897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72d26c27-747b-4cfd-91e1-08b8e56cf37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:17.527219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 840/1661] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:17.538015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:17.559302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f279e6f-0f87-4be2-b2ea-cefbaaedee28 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:17.576192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:17.576846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:17.577129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:17.577403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:17.596788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:17.598463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd8b4-87d3-4a95-88ba-cdd4889fc3ee service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:17.628537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 822/1662] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:19.596821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be67f-bc47-4804-b53c-19d0d3afdf53 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:19.597579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be67f-bc47-4804-b53c-19d0d3afdf53 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:19.597579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be67f-bc47-4804-b53c-19d0d3afdf53 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:19.598926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 841/1663] 10.222.0.47 () {54 vars in 912 bytes} [Mon Aug 1 22:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:19.606774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f314-a7ae-4659-849b-8b0f278edc9c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:19.607332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f314-a7ae-4659-849b-8b0f278edc9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:19.607551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f314-a7ae-4659-849b-8b0f278edc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:19.712079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0733f314-a7ae-4659-849b-8b0f278edc9c None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:19.718210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0733f314-a7ae-4659-849b-8b0f278edc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:19.724591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:19.724879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:19.725154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:19.775979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] MFA Rules not processed for user `a2ff5f41f9b44c2c9f5f90f13c266fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:19.780177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:19.836841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:19.890025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 823/1664] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:20.019337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 842/1665] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:20.032175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:20.032831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:20.033237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:20.083030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] MFA Rules not processed for user `a2ff5f41f9b44c2c9f5f90f13c266fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:20.084345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:20.093256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 824/1666] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:20.128679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:20.136761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:20.160495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43bc19-a9ce-4733-af64-d4b840e13166 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:20.186768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:20.186943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:20.187170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:20.187439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:20.209376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:20.211087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcabfe8-b2e9-4385-bd60-3e250ae970e1 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:20.254663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 843/1667] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:20.389174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:20.402074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712755b-fd2d-4ba2-b301-4b5fba1784fc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:20.416284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:20.417909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:20.418263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:20.420070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:20.450052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:20.452363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fad5cf-3bbd-4f23-a513-16d93bbb5b3a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:20.475389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 825/1668] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:21.405473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f12dc-e837-43f4-a357-e85b7522c6fb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:21.406015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f12dc-e837-43f4-a357-e85b7522c6fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:21.406015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f12dc-e837-43f4-a357-e85b7522c6fb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:21.407477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 844/1669] 10.222.0.47 () {54 vars in 912 bytes} [Mon Aug 1 22:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:21.414041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2083-05e5-46ee-8646-de0256cd9713 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:21.414912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2083-05e5-46ee-8646-de0256cd9713 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:21.415320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2083-05e5-46ee-8646-de0256cd9713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:21.507026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-367d2083-05e5-46ee-8646-de0256cd9713 None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:21.509152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-367d2083-05e5-46ee-8646-de0256cd9713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:21.528311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 826/1670] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:21.548463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f12dc-e837-43f4-a357-e85b7522c6fb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:21.562519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f12dc-e837-43f4-a357-e85b7522c6fb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:21.580500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:21.581350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:21.581906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:21.582349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:21.597475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:21.601259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9db35-47ff-4e98-a925-d0496198af12 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:21.633315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 845/1671] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:21.670034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:21.670627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:21.670901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:21.758941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] MFA Rules not processed for user `e6d336332a65408c8a438c8f96f4415d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:21.763016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:22.092591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d865d85-c5dd-41da-813f-c2906ba18597 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:22.092882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d865d85-c5dd-41da-813f-c2906ba18597 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:22.093341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d865d85-c5dd-41da-813f-c2906ba18597 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:22.096580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 846/1672] 10.222.0.47 () {54 vars in 961 bytes} [Mon Aug 1 22:43:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:22.103471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b917-e81b-4727-8704-f794898994d8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:22.104412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b917-e81b-4727-8704-f794898994d8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:22.105030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b917-e81b-4727-8704-f794898994d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:22.114777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 827/1673] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:22.151187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0406b917-e81b-4727-8704-f794898994d8 None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:22.156250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0406b917-e81b-4727-8704-f794898994d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:22.382069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 847/1674] 10.222.0.47 () {58 vars in 1058 bytes} [Mon Aug 1 22:43:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:22.403638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:22.422060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16855e5-5d04-482a-96ff-5c0d9add3d58 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:22.442998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:22.446763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:22.447695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:22.448587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:22.494252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:22.527167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:22.531099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162956b0-1414-4669-a6e9-224fa3df8610 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:22.586985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 828/1675] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:22.631121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406b917-e81b-4727-8704-f794898994d8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:22.634461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:22.634690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:22.634963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:22.669737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406b917-e81b-4727-8704-f794898994d8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:22.694966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:22.695750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:22.696055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:22.697324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:22.698839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] MFA Rules not processed for user `a8bbeb0e65a54b1199d0fe718168af70`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:22.703257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:22.725104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:22.728557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fbbd9-8c64-47b9-8aa0-514551b4e8f1 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:22.735782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 829/1676] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:22.772482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 848/1677] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:23.084479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:23.098665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47118e4d-f532-4f5f-bb8a-d5aaa749fa23 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:23.131562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:23.132225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:23.132497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:23.132766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:23.337633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:23.342842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:23.375949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 830/1678] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:23.536963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710166-fcca-4302-b97d-120fee91addd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:23.537490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710166-fcca-4302-b97d-120fee91addd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:23.537973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710166-fcca-4302-b97d-120fee91addd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:23.587807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a2710166-fcca-4302-b97d-120fee91addd None None] MFA Rules not processed for user `527b7873931742049db9f44104396686`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:23.594908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a2710166-fcca-4302-b97d-120fee91addd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:23.830311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 849/1679] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:24.666450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:24.720153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77555-29b2-4636-b1c3-6b9fb7e06eb4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:24.752905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:24.759456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:24.763605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:24.766993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:24.816677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:24.820771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf0f8-a27a-4236-a4a7-5e2a81b2e0a6 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:24.870055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 831/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:25.896427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710166-fcca-4302-b97d-120fee91addd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:26.091221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:26.125769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710166-fcca-4302-b97d-120fee91addd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:26.142902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:26.143852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:26.144275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:26.144687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] PATH_INFO: `/v3/users/8f6f6e5693dc4f0aa985b6313b6ffcbe` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:26.146391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6f6e5693dc4f0aa985b6313b6ffcbe)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:26.148284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:26.148284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Aug 01 22:43:26.148284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: Aug 01 22:43:26.150008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:26.326554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6f6e5693dc4f0aa985b6313b6ffcbe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:26.343587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6f6e5693dc4f0aa985b6313b6ffcbe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:26.346240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6f6e5693dc4f0aa985b6313b6ffcbe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:26.347460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6f6e5693dc4f0aa985b6313b6ffcbe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:26.415439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6f6e5693dc4f0aa985b6313b6ffcbe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:26.461110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 850/1681] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:25 2022] DELETE /identity/v3/users/8f6f6e5693dc4f0aa985b6313b6ffcbe => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:26.465918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269876e-40ef-4041-a78c-f31c2f9d513e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:26.466780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269876e-40ef-4041-a78c-f31c2f9d513e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:26.467631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269876e-40ef-4041-a78c-f31c2f9d513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:26.516513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0269876e-40ef-4041-a78c-f31c2f9d513e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:26.520628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0269876e-40ef-4041-a78c-f31c2f9d513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:26.572880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:26.701740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 832/1682] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:26.726945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:26.729155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:26.899213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0aef6-7691-4489-a489-686dcd4593c2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:26.910196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:26.910883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:26.911166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:26.911439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:26.947632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:26.950099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:26.996481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 851/1683] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:27.722888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435597dc-37d2-469f-8993-b23c3c1f8018 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:27.723450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435597dc-37d2-469f-8993-b23c3c1f8018 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:27.723526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435597dc-37d2-469f-8993-b23c3c1f8018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:27.886198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-435597dc-37d2-469f-8993-b23c3c1f8018 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:27.888831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-435597dc-37d2-469f-8993-b23c3c1f8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:28.101353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:28.107582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:28.138995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1bc45-e614-4244-b4f1-237b68c206c7 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:28.164273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:28.164963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:28.165244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:28.165516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] PATH_INFO: `/v3/users/e847171f82374d59bb01616636385a23` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:28.171370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e847171f82374d59bb01616636385a23)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:28.176272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:28.248624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 833/1684] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:28.407616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e847171f82374d59bb01616636385a23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:28.434070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e847171f82374d59bb01616636385a23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:28.440756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e847171f82374d59bb01616636385a23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:28.444288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e847171f82374d59bb01616636385a23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:28.473188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:28.494433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e847171f82374d59bb01616636385a23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:28.566766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 852/1685] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:28 2022] DELETE /identity/v3/users/e847171f82374d59bb01616636385a23 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:28.569393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f083-3304-4442-b136-8968fade516d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:28.570403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f083-3304-4442-b136-8968fade516d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:28.570859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f083-3304-4442-b136-8968fade516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:28.649009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6919f083-3304-4442-b136-8968fade516d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:28.653508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6919f083-3304-4442-b136-8968fade516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:28.656179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:28.657068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:28.922328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 834/1686] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:28.966580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f099f-e309-40d7-abc1-2751446f362e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:28.984507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:28.985398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:28.985852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:28.986386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:28.991446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:29.025449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f083-3304-4442-b136-8968fade516d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:29.027717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6919f083-3304-4442-b136-8968fade516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:29.129875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:29.181124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:29.183773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a408087-4190-4d22-aef6-3f6c8902b32e service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:29.201363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 853/1687] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:43:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:43:29.224232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f083-3304-4442-b136-8968fade516d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:29.247125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:29.249150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:29.250316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:29.252861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:29.289158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:29.294778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:29.404541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 835/1688] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:29.629326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38cb8-7be6-4ad4-ab0d-5dce06448456 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:29.631226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38cb8-7be6-4ad4-ab0d-5dce06448456 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:29.631615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38cb8-7be6-4ad4-ab0d-5dce06448456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:29.693797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a1e38cb8-7be6-4ad4-ab0d-5dce06448456 None None] MFA Rules not processed for user `e6d336332a65408c8a438c8f96f4415d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:29.706701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a1e38cb8-7be6-4ad4-ab0d-5dce06448456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:29.922425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 854/1689] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:31.302112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:31.304054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:31.356349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655f7cd-b95a-42d8-99af-5d8a28cb52e6 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:31.400818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:31.405984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:31.406502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:31.406993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] PATH_INFO: `/v3/projects/fcf1ca7cb1194e0ca37da94225a1e139` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:31.416201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf1ca7cb1194e0ca37da94225a1e139)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:31.421539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 22:43:31.421539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Aug 01 22:43:31.421539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: Aug 01 22:43:31.427151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:31.689897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf1ca7cb1194e0ca37da94225a1e139', 'request_id': 'req-fc89b8ad-72a2-49ff-a6c6-697cca219dad'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:31.709190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf1ca7cb1194e0ca37da94225a1e139', 'request_id': 'req-fc89b8ad-72a2-49ff-a6c6-697cca219dad'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:31.726958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf1ca7cb1194e0ca37da94225a1e139', 'request_id': 'req-fc89b8ad-72a2-49ff-a6c6-697cca219dad'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:31.794000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] The token cache is being invalidate because project fcf1ca7cb1194e0ca37da94225a1e139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:31.794364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:31.798185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc89b8ad-72a2-49ff-a6c6-697cca219dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:31.800951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 836/1690] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:31 2022] DELETE /identity/v3/projects/fcf1ca7cb1194e0ca37da94225a1e139 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:31.806303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:31.807022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:31.807022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:31.849774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:31.854663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:31.872911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:32.052146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 855/1691] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:32.060990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.060990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.060990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.114954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:32.116241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:32.147702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 837/1692] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:32.160898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.179281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dbaf6-6a15-49df-b507-c6884611c818 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:32.206900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:32.207872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.208369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.208849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.209831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:32.210357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:32.216611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:32.262635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 856/1693] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:32.263914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.296093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292f808-7d4c-4cde-aef6-30272edad8c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:32.332214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:32.333223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.333223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.333223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.333539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:32.351650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:32.461343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 838/1694] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:32 2022] POST /identity/v3/projects => generated 401 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:32.466516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.494139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac344c-afcb-4945-bba3-faea2ef34a5d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:32.706664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:32.707324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.707603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.707883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.708573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:32.710598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:32.790939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 857/1695] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:32 2022] POST /identity/v3/users => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:32.795814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.811901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b596226-6de2-43ab-b254-5bc481790aac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:32.832742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:32.833370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.834585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.836201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.836201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:32.841279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31b757-6763-40d1-af28-0a04356d8a52 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:32.870813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 839/1696] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:43:32.872448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.884532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ebd94-b1b3-4cbb-a6e5-353c43ae833e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.887208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714a28d-79dc-4544-a3a8-9ad415fc2e1f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:32.887769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ebd94-b1b3-4cbb-a6e5-353c43ae833e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.888229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ebd94-b1b3-4cbb-a6e5-353c43ae833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.907135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:32.912867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:32.915257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:32.915802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:32.917751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:32.924472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:32.958708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 858/1697] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:43:32.961838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5f8ebd94-b1b3-4cbb-a6e5-353c43ae833e None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:32.970621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:32.971760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5f8ebd94-b1b3-4cbb-a6e5-353c43ae833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.022410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8a43c-334c-4dbc-8ccf-318b0b8b4d20 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:33.042987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:33.044054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.047511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.049007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] PATH_INFO: `/v3/projects/d37c08e143124925af12eb062b11d224/users/54ce4988e6cf4f16ba2df40a7d7e1ac0/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.114045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37c08e143124925af12eb062b11d224, user_id=54ce4988e6cf4f16ba2df40a7d7e1ac0, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:33.122653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:33.205717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 840/1698] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 1 22:43:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:33.218712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 859/1699] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:32 2022] PUT /identity/v3/projects/d37c08e143124925af12eb062b11d224/users/54ce4988e6cf4f16ba2df40a7d7e1ac0/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:33.236136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.236567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.236973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.270387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:33.272259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.323336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] MFA Rules not processed for user `54ce4988e6cf4f16ba2df40a7d7e1ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:33.327061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.370388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:33.453511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a501-1dac-4296-a0f4-45ce2fa0011c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:33.462411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:33.465418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.465959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.466072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.497606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:33.512603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:33.515376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:33.539666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 860/1700] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:43:33.540609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:33.545164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.558143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 841/1701] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:33.559202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:33.563248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.599484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69955bc-7993-4423-9390-f972b25ef47e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:33.608247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:33.612540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.613048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.614090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.657746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:33.659233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:33.697626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 842/1702] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:33.700348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:33.738532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8ca0-31b0-40db-8085-f1fc4c51fe00 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:33.745089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:33.745878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.746530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.747374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] PATH_INFO: `/v3/projects/b32a8358a242425db77284978619d416` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.749374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32a8358a242425db77284978619d416)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:33.751704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:33.805771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40f49-0701-46c5-ac54-ad1a0d38cf1d service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:33.813122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:33.814055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.814480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.814703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.815750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:33.818936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29a37-e5e5-4e53-bd12-5a632c2cd72c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:33.853295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 843/1703] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:33 2022] POST /identity/v3/projects => generated 405 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:33.854996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990d31-5000-4f76-960b-66469777146e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:33.855410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990d31-5000-4f76-960b-66469777146e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:33.855760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990d31-5000-4f76-960b-66469777146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:33.879036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-52990d31-5000-4f76-960b-66469777146e None None] MFA Rules not processed for user `527b7873931742049db9f44104396686`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:33.882475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52990d31-5000-4f76-960b-66469777146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:33.889453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32a8358a242425db77284978619d416', 'request_id': 'req-15d14f4c-2eb8-4780-90bc-5562929a370a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:33.912083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32a8358a242425db77284978619d416', 'request_id': 'req-15d14f4c-2eb8-4780-90bc-5562929a370a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:33.932561 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32a8358a242425db77284978619d416', 'request_id': 'req-15d14f4c-2eb8-4780-90bc-5562929a370a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:33.965756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 844/1704] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:33.965756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52990d31-5000-4f76-960b-66469777146e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:33.986339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] The token cache is being invalidate because project b32a8358a242425db77284978619d416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:33.986707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:33.988527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.000673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 861/1705] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:33 2022] DELETE /identity/v3/projects/b32a8358a242425db77284978619d416 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:34.009826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:34.013440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:34.086983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:34.205223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52990d31-5000-4f76-960b-66469777146e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:34.236867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:34.243420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:34.246344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:34.246769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:34.249480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:34.263164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:34.322507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d14f4c-2eb8-4780-90bc-5562929a370a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:34.358158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:34.361819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:34.367411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 845/1706] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:33 2022] POST /identity/v3/users => generated 309 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:34.367623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:34.367623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] PATH_INFO: `/v3/users/784f128bbb91461aa0fb7ad098b8284e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:34.384643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:34.386257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784f128bbb91461aa0fb7ad098b8284e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:34.389882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:34.392725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:34.457888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8207e7-46d3-4c76-bac2-ac57ccaac017 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:34.478532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:34.482379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:34.495107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:34.497342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:34.499556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:34.505705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:34.540558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:34.587729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 846/1707] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:34.608948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:34.652641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab5cf-6dcb-41dd-81d6-46425c9bdc0f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:34.667716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:34.668762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:34.668892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784f128bbb91461aa0fb7ad098b8284e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.669294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:34.669785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] PATH_INFO: `/v3/projects/186cbd4178a248f6881c0b807bab1a9a/users/b2b6e3ebf9a54d82ae46d52440c3ee0e/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:34.718329 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784f128bbb91461aa0fb7ad098b8284e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.719211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784f128bbb91461aa0fb7ad098b8284e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.719993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784f128bbb91461aa0fb7ad098b8284e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.760575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784f128bbb91461aa0fb7ad098b8284e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:34.761039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186cbd4178a248f6881c0b807bab1a9a, user_id=b2b6e3ebf9a54d82ae46d52440c3ee0e, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:34.769061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:34.804599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 862/1708] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:34 2022] DELETE /identity/v3/users/784f128bbb91461aa0fb7ad098b8284e => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:34.827137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 847/1709] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:34 2022] PUT /identity/v3/projects/186cbd4178a248f6881c0b807bab1a9a/users/b2b6e3ebf9a54d82ae46d52440c3ee0e/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:34.833098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:34.835111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:34.975919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f8b7f-2e7e-4501-8bf9-7e2aa226f311 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:35.089343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:35.090188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:35.090575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:35.091208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:35.092393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:35.094881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53de06b-d3a2-4390-a16d-346998560665 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:35.114315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:35.116628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 863/1710] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:43:35.131635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaaf7d-4e0b-4e41-ad4c-eef8aab88f96 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:35.145752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:35.148644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:35.149056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:35.150082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] PATH_INFO: `/v3/projects/186cbd4178a248f6881c0b807bab1a9a/users/b2b6e3ebf9a54d82ae46d52440c3ee0e/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:35.155775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] RBAC: Authorizing `identity:create_grant(project_id=186cbd4178a248f6881c0b807bab1a9a, user_id=b2b6e3ebf9a54d82ae46d52440c3ee0e, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:35.162556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:35.213839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:35.214416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 848/1711] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:35 2022] PUT /identity/v3/projects/186cbd4178a248f6881c0b807bab1a9a/users/b2b6e3ebf9a54d82ae46d52440c3ee0e/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:35.214750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:35.215184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:35.309477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] MFA Rules not processed for user `b2b6e3ebf9a54d82ae46d52440c3ee0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:35.311260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:35.472080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:35.593518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 864/1712] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:35.653848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:35.656541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:36.020193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857aa57-31cc-4d4b-b233-1f39731c529d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:36.035601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:36.040723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:36.048098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:36.050190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:36.093028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:36.099664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa860ac7-ea00-4c60-99e3-6cc2d0adf2ed service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:36.210926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 849/1713] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:35 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:36.803032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:36.944864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094a24-c5cc-4cf5-8262-b91866d26d84 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:36.955850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:36.957028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:36.958012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:36.975378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] PATH_INFO: `/v3/users/54ce4988e6cf4f16ba2df40a7d7e1ac0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:36.977076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ce4988e6cf4f16ba2df40a7d7e1ac0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:36.979611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:37.077626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ce4988e6cf4f16ba2df40a7d7e1ac0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:37.104902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ce4988e6cf4f16ba2df40a7d7e1ac0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:37.106170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ce4988e6cf4f16ba2df40a7d7e1ac0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:37.107135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ce4988e6cf4f16ba2df40a7d7e1ac0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:37.139192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ce4988e6cf4f16ba2df40a7d7e1ac0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:37.222706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 865/1714] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:36 2022] DELETE /identity/v3/users/54ce4988e6cf4f16ba2df40a7d7e1ac0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:37.222907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d2fec-9852-446d-84f0-a3d4577735f1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:37.223436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d2fec-9852-446d-84f0-a3d4577735f1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:37.223601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d2fec-9852-446d-84f0-a3d4577735f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:37.280520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6e5d2fec-9852-446d-84f0-a3d4577735f1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:37.286280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6e5d2fec-9852-446d-84f0-a3d4577735f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:37.495146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 850/1715] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:37.543717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:37.545595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:37.559446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6668-0e71-4ecf-afcd-bd9d2b48714c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:37.559744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6668-0e71-4ecf-afcd-bd9d2b48714c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:37.560311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6668-0e71-4ecf-afcd-bd9d2b48714c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:37.563613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 851/1716] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:37.697874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635d39-53b5-4a51-9741-8fb5f8b37ec9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:37.698277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635d39-53b5-4a51-9741-8fb5f8b37ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:37.698508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635d39-53b5-4a51-9741-8fb5f8b37ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:37.759280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e0635d39-53b5-4a51-9741-8fb5f8b37ec9 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:37.764596 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e0635d39-53b5-4a51-9741-8fb5f8b37ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:37.820230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ece4c8-c544-4a74-9725-65a1da49e4f4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:37.839929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:37.845052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:37.847675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:37.849541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:37.851171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:37.898608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:37.928254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:37.938147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:38.017869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 866/1717] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:38.146944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:38.151836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:38.272144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 852/1718] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:43:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:38.294085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194adb97-0287-4349-a032-f1fe8c160e13 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:38.322352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:38.326174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:38.326671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:38.329969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] PATH_INFO: `/v3/projects/f5ef02eba6714a0795d77c55cf868241` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:38.337321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ef02eba6714a0795d77c55cf868241)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:38.347407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:38.696881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdd43-b9ed-42ad-974e-3f60995c3bac None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:38.697374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdd43-b9ed-42ad-974e-3f60995c3bac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:38.698131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdd43-b9ed-42ad-974e-3f60995c3bac None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:38.703642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 853/1719] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:38.741305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ef02eba6714a0795d77c55cf868241', 'request_id': 'req-73a33444-8a43-4f35-abb1-b2903a0cf550'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:38.743905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:38.744210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:38.744466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:38.765543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ef02eba6714a0795d77c55cf868241', 'request_id': 'req-73a33444-8a43-4f35-abb1-b2903a0cf550'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:38.806544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ef02eba6714a0795d77c55cf868241', 'request_id': 'req-73a33444-8a43-4f35-abb1-b2903a0cf550'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:38.826078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:38.828590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:38.852322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] The token cache is being invalidate because project f5ef02eba6714a0795d77c55cf868241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:38.857648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:38.858939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:38.867802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 867/1720] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:38 2022] DELETE /identity/v3/projects/f5ef02eba6714a0795d77c55cf868241 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:38.870059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:38.875571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:39.205998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a33444-8a43-4f35-abb1-b2903a0cf550 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:39.228806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:39.230395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:39.235060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:39.236397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] PATH_INFO: `/v3/users/56b163434a6a4b748e3b86343bcb9794` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:39.241531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b163434a6a4b748e3b86343bcb9794)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:39.252677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:39.289589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 854/1721] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 546 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:39.364740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:39.366790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:39.391922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96afd9-fc06-4e6e-aaaf-3f36fc783a06 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:39.477090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:39.483226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:39.484161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:39.485330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:39.498005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:39.513238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b163434a6a4b748e3b86343bcb9794'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:39.548529 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b163434a6a4b748e3b86343bcb9794'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:39.549741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b163434a6a4b748e3b86343bcb9794'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:39.551379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b163434a6a4b748e3b86343bcb9794'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:39.557838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:39.559495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:39.602414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 855/1722] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:43:39.611498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-58c6776b-8a70-46bd-ad0d-fa30995ce021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b163434a6a4b748e3b86343bcb9794'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:39.627316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:39.632949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:39.640008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:39.693343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 868/1723] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:38 2022] DELETE /identity/v3/users/56b163434a6a4b748e3b86343bcb9794 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:39.714620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfac6c-eb96-472b-9abd-0cb3768a7f36 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:39.893235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:39.914675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:39.914767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:39.915209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] PATH_INFO: `/v3/users/fad57ee650d8414eacceee3fa5bb9286` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:39.918285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad57ee650d8414eacceee3fa5bb9286)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:39.923541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:40.043324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:40.080616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad57ee650d8414eacceee3fa5bb9286'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:40.081460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad57ee650d8414eacceee3fa5bb9286'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:40.092100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad57ee650d8414eacceee3fa5bb9286'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:40.113829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad57ee650d8414eacceee3fa5bb9286'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:40.114519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad57ee650d8414eacceee3fa5bb9286'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:40.183229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 856/1724] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:39 2022] DELETE /identity/v3/users/fad57ee650d8414eacceee3fa5bb9286 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:40.189877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:40.190738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:40.191177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:40.236638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:40.238084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:40.437325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 869/1725] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:40.508606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:40.510157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:40.692081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ea6eb-35d3-4960-968d-5e3b543e0172 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:40.704407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:40.705286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:40.705701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:40.706076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:40.728595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:40.730324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:40.755019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 857/1726] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:40.758549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:40.760430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:40.793818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05585841-6b18-497a-8eca-e9f1d2e79572 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:40.817147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:40.817879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:40.818943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:40.832360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] PATH_INFO: `/v3/projects/d37c08e143124925af12eb062b11d224` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:40.837360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37c08e143124925af12eb062b11d224)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:40.852122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:40.919531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:40.921463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:41.045171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37c08e143124925af12eb062b11d224', 'request_id': 'req-c68bcc63-4470-4791-8dcc-d821dd7601e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.058608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37c08e143124925af12eb062b11d224', 'request_id': 'req-c68bcc63-4470-4791-8dcc-d821dd7601e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.069680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37c08e143124925af12eb062b11d224', 'request_id': 'req-c68bcc63-4470-4791-8dcc-d821dd7601e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.130425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] The token cache is being invalidate because project d37c08e143124925af12eb062b11d224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:41.130717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.134055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.139680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 870/1727] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:40 2022] DELETE /identity/v3/projects/d37c08e143124925af12eb062b11d224 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:41.152801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:41.155421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:41.169450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d97cb6-4f6e-4d22-9be4-b75bc3e638d8 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:41.328516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:41.359468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bcc63-4470-4791-8dcc-d821dd7601e7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:41.396236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:41.398957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:41.399919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:41.400825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:41.401382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:41.401914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] PATH_INFO: `/v3/users/b2b6e3ebf9a54d82ae46d52440c3ee0e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:41.402083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:41.402281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:41.403901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b6e3ebf9a54d82ae46d52440c3ee0e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:41.405099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:41.414027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:41.448970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:41.455568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:41.505027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:41.517591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 858/1728] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:41.606993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b6e3ebf9a54d82ae46d52440c3ee0e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.638059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b6e3ebf9a54d82ae46d52440c3ee0e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.638993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b6e3ebf9a54d82ae46d52440c3ee0e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.640246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b6e3ebf9a54d82ae46d52440c3ee0e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.694596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b6e3ebf9a54d82ae46d52440c3ee0e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:41.745208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 871/1729] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:41 2022] DELETE /identity/v3/users/b2b6e3ebf9a54d82ae46d52440c3ee0e => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:42.980448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:42.982133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.015866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3b36-95a5-483c-8267-1c20192e4442 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.106781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.108871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.189262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.190608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.190888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.191159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] PATH_INFO: `/v3/projects/7223873e6eac420abffcf2ea14e602ca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.192905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7223873e6eac420abffcf2ea14e602ca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.198593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.312923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1311c-02b5-48fd-86f5-73ea95071625 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.337240 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7223873e6eac420abffcf2ea14e602ca', 'request_id': 'req-f04ba957-1458-40b5-8743-0b8a74d7853d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.353692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7223873e6eac420abffcf2ea14e602ca', 'request_id': 'req-f04ba957-1458-40b5-8743-0b8a74d7853d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.385521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7223873e6eac420abffcf2ea14e602ca', 'request_id': 'req-f04ba957-1458-40b5-8743-0b8a74d7853d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.415234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] The token cache is being invalidate because project 7223873e6eac420abffcf2ea14e602ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:43.415657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.416434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.419101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 859/1730] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:42 2022] DELETE /identity/v3/projects/7223873e6eac420abffcf2ea14e602ca => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:43.419780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.421868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.465337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.466139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.466481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.467142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] PATH_INFO: `/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.468518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57bcb4f3eb4428b80d64858b6a34ad2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.470660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.526869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:43.528384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57bcb4f3eb4428b80d64858b6a34ad2', 'request_id': 'req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.531234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ba957-1458-40b5-8743-0b8a74d7853d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.540643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57bcb4f3eb4428b80d64858b6a34ad2', 'request_id': 'req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.546317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:43.552132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57bcb4f3eb4428b80d64858b6a34ad2', 'request_id': 'req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.566995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] The token cache is being invalidate because project c57bcb4f3eb4428b80d64858b6a34ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:43.567303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.567959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.569663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 872/1731] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:43 2022] DELETE /identity/v3/projects/c57bcb4f3eb4428b80d64858b6a34ad2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:43.570474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.571813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.608245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.609001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.609346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.609769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] PATH_INFO: `/v3/projects/186cbd4178a248f6881c0b807bab1a9a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.611149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186cbd4178a248f6881c0b807bab1a9a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.613086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.653471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa15c-749e-47d0-b2b7-0c885fa28f2b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.659057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.659730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.660010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.660283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] PATH_INFO: `/v3/users/9a11e8c6fb0144919b9d511f72fba6fc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.661513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a11e8c6fb0144919b9d511f72fba6fc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.663517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.685449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186cbd4178a248f6881c0b807bab1a9a', 'request_id': 'req-64e91108-f29c-44d4-8c03-0dc3d32cf246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.694941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186cbd4178a248f6881c0b807bab1a9a', 'request_id': 'req-64e91108-f29c-44d4-8c03-0dc3d32cf246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.702276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186cbd4178a248f6881c0b807bab1a9a', 'request_id': 'req-64e91108-f29c-44d4-8c03-0dc3d32cf246'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.726057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a11e8c6fb0144919b9d511f72fba6fc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.727992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] The token cache is being invalidate because project 186cbd4178a248f6881c0b807bab1a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:43.728784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.729957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-64e91108-f29c-44d4-8c03-0dc3d32cf246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.733436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 860/1732] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:43 2022] DELETE /identity/v3/projects/186cbd4178a248f6881c0b807bab1a9a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:43.735571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa57b-f3f3-469e-880d-58696c5db5fa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.736388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa57b-f3f3-469e-880d-58696c5db5fa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.736822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa57b-f3f3-469e-880d-58696c5db5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.741363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a11e8c6fb0144919b9d511f72fba6fc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.742236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a11e8c6fb0144919b9d511f72fba6fc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.743389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a11e8c6fb0144919b9d511f72fba6fc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.760952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9836689e-dca4-4a1a-ab7a-884ff4d8d61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a11e8c6fb0144919b9d511f72fba6fc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:43.766259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-413fa57b-f3f3-469e-880d-58696c5db5fa None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:43.768315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-413fa57b-f3f3-469e-880d-58696c5db5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.778923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 873/1733] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:43 2022] DELETE /identity/v3/users/9a11e8c6fb0144919b9d511f72fba6fc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:43.780946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5179b3-dc1c-42db-bbe6-d34fd2b46e01 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.781141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5179b3-dc1c-42db-bbe6-d34fd2b46e01 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.781428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5179b3-dc1c-42db-bbe6-d34fd2b46e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.807273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ae5179b3-dc1c-42db-bbe6-d34fd2b46e01 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:43.808054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ae5179b3-dc1c-42db-bbe6-d34fd2b46e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.862088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 861/1734] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:43.871178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.871508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.871802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.883639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:43.899583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:43.900956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.906421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 874/1735] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:43.909675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.910195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.910559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 862/1736] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:43:43.910787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.914949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.924946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4545cd-8dae-4d21-9fa6-4c4caed41fff None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.931037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.931741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.932052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.932448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.933510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:43.933510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.935532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.943757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:43.945357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:43.953877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 863/1737] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:43:43.954495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 875/1738] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:43.955695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.959484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:43.968273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a7536-5008-4a78-a101-69f3e2388975 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.969803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0eae-e422-4522-9641-cd72291a8729 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:43.974590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.975322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.975658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.975848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:43.976006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.976490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:43.976774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:43.977001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:43.977110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:43.977376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.977752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:43.979317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.979736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:43.993783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 876/1739] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:43.998074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.012862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000101-565d-4958-aee7-88fdf6c4981d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.016632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 864/1740] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:43:44.019489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.020606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.020606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.020883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.021053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.021368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.023507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.027854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:44.063170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 877/1741] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:43 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:43:44.066625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.096258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04c33-a297-4d27-b2c7-ec548489a8b1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.144360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834836b-8a4a-41d2-8864-e34cd7682f9e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.154148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.156719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.157007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.157274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.158024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.160342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.185446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.185712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 878/1742] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:44.186559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.186977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.187099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.187419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.188375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.190723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.204287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304121-218b-41a8-bd6c-b68a4d326bc3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.204492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.209940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.210854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.211206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.211537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] PATH_INFO: `/v3/projects/ef6d7852e147412784f42d75031ffe59` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.212842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6d7852e147412784f42d75031ffe59)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.215474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.259931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 865/1743] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/users => generated 305 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:44.259931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.283197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258512-55ac-47dc-886d-a3a8385e8875 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.313180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6d7852e147412784f42d75031ffe59', 'request_id': 'req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:44.329129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6d7852e147412784f42d75031ffe59', 'request_id': 'req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:44.342516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6d7852e147412784f42d75031ffe59', 'request_id': 'req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:44.364824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] The token cache is being invalidate because project ef6d7852e147412784f42d75031ffe59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:44.365116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:44.365810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:44.370900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.371414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 879/1744] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:44 2022] DELETE /identity/v3/projects/ef6d7852e147412784f42d75031ffe59 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:44.371813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.372248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.372452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.372808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.374081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.374747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.377614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70dbe9-4e51-4556-9c43-aaf06ae04c4f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.387760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce822f1-ab97-400e-bdc3-c6688d2a9a5d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.392824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.393892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.394157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 866/1745] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:44.394318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.394597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.395749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.396210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.396588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.396957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.397063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.409776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 880/1746] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:44.410371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.412327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.429601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb2829-47f2-4868-be3b-61961ae778e9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.436065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.436723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.437003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.437275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.438660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.440375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.445905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:44.449227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.458846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 881/1747] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:44.462939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.470966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:44.476326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6807c-30af-460c-a8fd-e4fb51cff663 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.494611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.494611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.494611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.494611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.495104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.495104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f43f90-ee95-4276-8ec6-2c1a0d1a8498 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.495386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 867/1748] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:44.495386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.512888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 882/1749] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:44.512888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.512888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.512888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.514111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d093e7e-c841-4ed7-a7fd-9403084e28e8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.523200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.524038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.524326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.524601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] PATH_INFO: `/v3/projects/bb263f293f65482c9ab680cb49dc3837/users/291893f8c2f34e44824275bfb89e6edb/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.534215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:44.535766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.546710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb263f293f65482c9ab680cb49dc3837, user_id=291893f8c2f34e44824275bfb89e6edb, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.549828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.557212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 883/1750] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:44.558275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.581617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c5b7-9403-4ad6-bbd0-d8a8f0c3afcf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.582459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 868/1751] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:44 2022] PUT /identity/v3/projects/bb263f293f65482c9ab680cb49dc3837/users/291893f8c2f34e44824275bfb89e6edb/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:44.584054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.597026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:44.659539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.660371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.660702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.661133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] PATH_INFO: `/v3/projects/c3f478bfe3b34203bcb7a75cfc134563/users/3fe0b65eff7b4fb58e642c19ac786b97/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.669083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa894eb-804b-4c2a-b154-812deb6f0eae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.675009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.675846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.676156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.676338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.677051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:44.677355 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.679022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadfa7-e993-42ae-a82a-97bcb96558e8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.687245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f478bfe3b34203bcb7a75cfc134563, user_id=3fe0b65eff7b4fb58e642c19ac786b97, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.690032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.694474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 869/1752] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:44.696459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547499a-994d-4a68-bdc1-c6c403351861 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.696826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547499a-994d-4a68-bdc1-c6c403351861 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.697102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547499a-994d-4a68-bdc1-c6c403351861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.706585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 884/1753] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:44 2022] PUT /identity/v3/projects/c3f478bfe3b34203bcb7a75cfc134563/users/3fe0b65eff7b4fb58e642c19ac786b97/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:44.707484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.727950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f547499a-994d-4a68-bdc1-c6c403351861 None None] MFA Rules not processed for user `291893f8c2f34e44824275bfb89e6edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:44.729217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f547499a-994d-4a68-bdc1-c6c403351861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.771968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e7fca-547f-47a3-aa63-23bc2049b267 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.776610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.777239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.777513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.777818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.778490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.780401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.797297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 870/1754] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:44.798734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.799025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.799327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.815077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 885/1755] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:44.815077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.830038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] MFA Rules not processed for user `3fe0b65eff7b4fb58e642c19ac786b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:44.831294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:44.879510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:44.883812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd44e3-fa79-4b19-8ff8-7fb046644401 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.892005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.893347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.894767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.895266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.896117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.898102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.914182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 871/1756] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:44.915173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.927209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdd91-855d-4319-893f-336eb0b94722 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:44.932225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:44.933004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:44.933347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:44.933827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:44.934690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:44.938186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 886/1757] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/projects => generated 401 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:44.938186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.938377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:44.966262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 872/1758] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/users => generated 283 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:44.966262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:44.970010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:44.981601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b73bb-88e0-46c8-8fee-b0889a93837f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.042336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393487e-9f12-46fc-b4ab-c48377f4a32c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.047243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.047881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.048159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.048430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.049141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.051145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.056636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.057318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.057651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.057931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.058640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.060589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.086564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 873/1759] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/users => generated 307 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:45.086761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 887/1760] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:44 2022] POST /identity/v3/projects => generated 407 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:45.086855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.088275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.096579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c723-733c-458b-b234-e723898205c3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.153181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee903e4f-abd8-458a-be87-3ddbbaec6157 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.155893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.156533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.156814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.157149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.157899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.158141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.158768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.159047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.159320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.159515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.160017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.161670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.170354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 874/1761] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:45.171480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.175522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 888/1762] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:45.178650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.185225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9051ad-77d6-46eb-9e42-0b1e89610e95 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.189284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef538-b599-46fa-800a-56f014b65547 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.190516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.191224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.191510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.191787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.192470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.193923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.194456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.194637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.194916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.195189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] PATH_INFO: `/v3/projects/17c9de17e8cb47a3afae367f17e262fe/users/0f0b28a6c85f442997b06665e8c8b7ee/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.211993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c9de17e8cb47a3afae367f17e262fe, user_id=0f0b28a6c85f442997b06665e8c8b7ee, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.215156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.217754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 875/1763] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:45 2022] POST /identity/v3/users => generated 310 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:45.218614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.229178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7924-5fe5-4f1d-914a-0b067978281b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.233331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 889/1764] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:45 2022] PUT /identity/v3/projects/17c9de17e8cb47a3afae367f17e262fe/users/0f0b28a6c85f442997b06665e8c8b7ee/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:45.234296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.279099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:45.295564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.296235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.296509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.296975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.297739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.300039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.301947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5857e-9dac-47c0-be57-8384685e146e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.307269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.308046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.308321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.308584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.309732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.311175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.311992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 876/1765] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:45.312948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.323011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 890/1766] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:45.323991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62640ef-97d1-436a-9472-aa01e0028a96 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.324148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.329028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.329703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.329978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.330248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.330956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.332591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.334334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:45.334873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ade5a8-0d6e-410d-b6aa-a3a6e62280f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.340949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.341625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.341915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.342237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] PATH_INFO: `/v3/projects/78079e342d0d4c1e8c9b267ab68a515e/users/177eadcf11b44bb385e3e21824ae324a/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.349414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 877/1767] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:45.350440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.360857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78079e342d0d4c1e8c9b267ab68a515e, user_id=177eadcf11b44bb385e3e21824ae324a, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.361086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79eee-2f02-4f9d-a0e5-6365f8ee39b8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.364718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.367379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.368232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.368425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.368635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] PATH_INFO: `/v3/projects/4aa73d9a8d184a47b5981476e6c52be0/users/8d1ebf53126446d49920625730e1c9ea/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.386114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa73d9a8d184a47b5981476e6c52be0, user_id=8d1ebf53126446d49920625730e1c9ea, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.386548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 891/1768] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:45 2022] PUT /identity/v3/projects/78079e342d0d4c1e8c9b267ab68a515e/users/177eadcf11b44bb385e3e21824ae324a/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:45.387214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.392371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a218d-809e-4fdb-a3f2-d54eca2ec706 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.422171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 878/1769] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:45 2022] PUT /identity/v3/projects/4aa73d9a8d184a47b5981476e6c52be0/users/8d1ebf53126446d49920625730e1c9ea/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:45.422171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.422171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.422515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.463809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] MFA Rules not processed for user `177eadcf11b44bb385e3e21824ae324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:45.465805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:45.537788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a1df-dd3b-4ecd-8146-11f1b992e617 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.662106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 879/1770] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:45.662106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.747909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.748653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.748933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.749214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] PATH_INFO: `/v3/projects/17c9de17e8cb47a3afae367f17e262fe/users/0f0b28a6c85f442997b06665e8c8b7ee/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.753433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c9de17e8cb47a3afae367f17e262fe, user_id=0f0b28a6c85f442997b06665e8c8b7ee, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.757892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:45.780090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 892/1771] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:45 2022] PUT /identity/v3/projects/17c9de17e8cb47a3afae367f17e262fe/users/0f0b28a6c85f442997b06665e8c8b7ee/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:45.780973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:45.877915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682ff18-2b02-4049-b34a-890d8f877802 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.907300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:45.935598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b8b4d-b3a6-48f2-b5ea-c5558b044afa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:45.958309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:45.959283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:45.962265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:45.963197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:45.966232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:45.968332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966ebd7-9221-4d07-8b75-9497b6b1a66d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.019978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 893/1772] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:45 2022] POST /identity/v3/projects => generated 363 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:46.021444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.021753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.022023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.037205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.039425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.041790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.054423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.059399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.060599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] MFA Rules not processed for user `0f0b28a6c85f442997b06665e8c8b7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:46.061881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:46.063243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.082191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 880/1773] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:46.082191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.086511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:46.095765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec3d8e-342f-4d0b-ac17-e058aa686d37 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.172903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 894/1774] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:46.174297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.214659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.214659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.214951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.215291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.216066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.218166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.253055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 881/1775] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:46 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:46.253768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.258155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:46.316787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea5217-cd6c-494a-ac64-dd8b2806f529 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.323270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.324430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.325598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.325598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] PATH_INFO: `/v3/projects/4aa73d9a8d184a47b5981476e6c52be0/users/8d1ebf53126446d49920625730e1c9ea/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.329771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa73d9a8d184a47b5981476e6c52be0, user_id=8d1ebf53126446d49920625730e1c9ea, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.334153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.353206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 895/1776] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:46 2022] PUT /identity/v3/projects/4aa73d9a8d184a47b5981476e6c52be0/users/8d1ebf53126446d49920625730e1c9ea/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:46.354642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.389258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706a57c-c873-457b-a9e8-c68f84bd5359 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.455085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.455745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.456256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.456385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.473492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:46.494223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e7d19-4c06-458e-8e84-e848e3d5d2a2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.500128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.500128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.500370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.500712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.501863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.505414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ea28-1234-45f5-b9d0-fb58f5c32d62 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.524172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 896/1777] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:46.525938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.526539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.526880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.558452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] MFA Rules not processed for user `8d1ebf53126446d49920625730e1c9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:46.559673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:46.579072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.580923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.587593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:46.594311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 882/1778] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:46.596295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.610060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718a400-4c90-4d80-b306-80f29acbcfb1 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.617595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.618568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.618953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.619331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] PATH_INFO: `/v3/projects/3229c75f14f74a38a2bff23cec529699/users/6f58776839594e7d9ab7f58099f7aacf/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.638957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3229c75f14f74a38a2bff23cec529699, user_id=6f58776839594e7d9ab7f58099f7aacf, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.643156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.647217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 897/1779] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:46.660799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.664283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 883/1780] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:46 2022] PUT /identity/v3/projects/3229c75f14f74a38a2bff23cec529699/users/6f58776839594e7d9ab7f58099f7aacf/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:46.670086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.677170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1389-c680-4dac-ae2a-c45af877a211 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.753742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.754590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cbe2f-2429-429d-85ea-0bc6422f7fbc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.754974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.755456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.755926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.759997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.760713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.761149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.761426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.762181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.763871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.779060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 884/1781] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:46.779060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.780895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:46.794777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3deaea1-60fc-45ce-8e1b-4af5c98b3766 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.798999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.799679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.799981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.800227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.844160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.846005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.858498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 898/1782] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:46.859315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:46.874504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1d034-01f2-4db5-b647-c6fda0b43664 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:46.875738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.877360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301af696-5c7a-4ca9-aa7b-1aa2e0666ec0 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.879514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:46.880216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.880505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.880775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] PATH_INFO: `/v3/projects/3229c75f14f74a38a2bff23cec529699/users/6f58776839594e7d9ab7f58099f7aacf/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.883016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3229c75f14f74a38a2bff23cec529699, user_id=6f58776839594e7d9ab7f58099f7aacf, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:46.885846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:46.888316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 885/1783] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:46.902153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 899/1784] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:46 2022] PUT /identity/v3/projects/3229c75f14f74a38a2bff23cec529699/users/6f58776839594e7d9ab7f58099f7aacf/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:46.913750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:46.914129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:46.914434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:46.952717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] MFA Rules not processed for user `6f58776839594e7d9ab7f58099f7aacf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:46.954179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:47.071454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:47.098881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 886/1785] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:47.139806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:47.161725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:47.220832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1c54-f736-477c-9553-504ec0e6ed8c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:47.225061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:47.225744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:47.226160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:47.226432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:47.237220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:47.242024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:47.250768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:47.256852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 900/1786] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:47.262748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5940e-bc5c-4002-ae8a-8907ecd96c49 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:47.267658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:47.268431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:47.268813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:47.269178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:47.366992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:47.369146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:47.381480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 887/1787] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:47.392621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:47.402524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8de64-e5b7-44d8-a4ea-2b37bb371ca5 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:47.406453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:47.407261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:47.407546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:47.407819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:47.470149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:47.470982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27f52f-da06-463a-a642-948933310dcf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:47.481600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 901/1788] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:55.868579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:55.959607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb24ed-7b35-4d69-97bc-2f16efca8aa0 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:55.968535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:55.969702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:55.970427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:55.971158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] PATH_INFO: `/v3/users/291893f8c2f34e44824275bfb89e6edb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:55.975100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291893f8c2f34e44824275bfb89e6edb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:55.979929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:56.111312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291893f8c2f34e44824275bfb89e6edb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.112975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291893f8c2f34e44824275bfb89e6edb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.127251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291893f8c2f34e44824275bfb89e6edb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.144498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291893f8c2f34e44824275bfb89e6edb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.145210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-954cee48-0824-4ab0-a8e2-cc06743e2709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291893f8c2f34e44824275bfb89e6edb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.187611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 888/1789] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:55 2022] DELETE /identity/v3/users/291893f8c2f34e44824275bfb89e6edb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:56.192256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.192806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.193004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.251229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:56.253472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:56.331717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 902/1790] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:56.337943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513dd-84b9-4c7b-a0b0-3f32f77baa40 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.338291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513dd-84b9-4c7b-a0b0-3f32f77baa40 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.338597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513dd-84b9-4c7b-a0b0-3f32f77baa40 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.340539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 889/1791] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:43:56.350483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.351343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.351771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.355683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:56.361116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:56.387494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:56.388885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:56.401917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:56.524214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3677-45fa-4e66-a4e4-d202e6699e22 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:56.536650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:56.538450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.538957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.539455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.544924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 890/1792] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:43:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:56.560736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:56.562520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:56.562835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:56.564671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:56.572048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:56.579762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 903/1793] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:56.585960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558946-d5e6-43c9-a11e-a48027377cec None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:56.594600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:56.595326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.595672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.596017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.607302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:56.609342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:56.621719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 891/1794] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:43:56.895673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:56.898420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:56.910567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c94c5-7ef2-496a-9a62-4257fe98190d service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:56.920105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:56.920814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:56.921090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:56.921357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] PATH_INFO: `/v3/projects/bb263f293f65482c9ab680cb49dc3837` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:56.922681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb263f293f65482c9ab680cb49dc3837)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:56.926736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:56.979450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb263f293f65482c9ab680cb49dc3837', 'request_id': 'req-937c7c47-9008-4a76-af10-734e312a402b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.988199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb263f293f65482c9ab680cb49dc3837', 'request_id': 'req-937c7c47-9008-4a76-af10-734e312a402b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:56.994276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb263f293f65482c9ab680cb49dc3837', 'request_id': 'req-937c7c47-9008-4a76-af10-734e312a402b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.010579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] The token cache is being invalidate because project bb263f293f65482c9ab680cb49dc3837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:57.010872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.011567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.013254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 904/1795] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:56 2022] DELETE /identity/v3/projects/bb263f293f65482c9ab680cb49dc3837 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:57.017352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:57.028373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c2464-9fb6-4781-bcdf-ea5fffafb6d5 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:57.088102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:57.088799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.089090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.089764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] PATH_INFO: `/v3/users/0f0b28a6c85f442997b06665e8c8b7ee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.090938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0b28a6c85f442997b06665e8c8b7ee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:57.093011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:57.141749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0b28a6c85f442997b06665e8c8b7ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.142409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0b28a6c85f442997b06665e8c8b7ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.149315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0b28a6c85f442997b06665e8c8b7ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.158059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0b28a6c85f442997b06665e8c8b7ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.158694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-82e788ad-3ff6-473e-aeb6-885ecc627792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0b28a6c85f442997b06665e8c8b7ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.178205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 892/1796] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:57 2022] DELETE /identity/v3/users/0f0b28a6c85f442997b06665e8c8b7ee => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:57.516150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:57.517697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:57.531797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c7c47-9008-4a76-af10-734e312a402b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:57.601954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:57.602619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.602900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.603175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] PATH_INFO: `/v3/projects/17c9de17e8cb47a3afae367f17e262fe` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.604401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c9de17e8cb47a3afae367f17e262fe)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:57.606878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:57.668943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c9de17e8cb47a3afae367f17e262fe', 'request_id': 'req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.697758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c9de17e8cb47a3afae367f17e262fe', 'request_id': 'req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.704603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c9de17e8cb47a3afae367f17e262fe', 'request_id': 'req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.717392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] The token cache is being invalidate because project 17c9de17e8cb47a3afae367f17e262fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:43:57.717841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.722173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2ac976f0-d80f-4bc6-9349-2c8508d9cb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:43:57.724316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 905/1797] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:43:57 2022] DELETE /identity/v3/projects/17c9de17e8cb47a3afae367f17e262fe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:57.729333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.729702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.730173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.750341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:57.751581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:57.757809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:57.817952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 893/1798] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:57.827219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.827557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.827836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.849787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:57.851042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:57.860114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 906/1799] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:57.864607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:57.874994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e1fdb-5e51-48f7-9d7d-7320b7f36ac7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:57.879952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:57.880593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.880871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.881144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.881895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:43:57.882200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:57.883830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:57.896614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 894/1800] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:43:57.901391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:57.907431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:57.911631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4218b90-6e35-465b-a323-75171e45b11c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:57.918599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:57.919339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:57.919911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:57.920190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:57.921157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:57.923261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:57.957324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 907/1801] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:57 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:57.962242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:57.974495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666313c5-e89f-45cd-85b9-ac2a5abe5a05 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.035599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.036332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.036555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.036805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.037528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.039519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.063696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 895/1802] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:57 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:58.068215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.079190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f1ce-0331-4618-9c57-cc8dffde81bc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.084072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.084705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.084985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.085259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.086019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.087633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.098779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 908/1803] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:58.103428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.114422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824832b-87f5-40d1-9595-666d7f66d9a3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.119654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.120186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.120454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.120745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.121583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.123114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7865b88-ff7b-403a-b111-d4cc7cdbae9b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.134857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 896/1804] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:43:58.139447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.149975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9b18c-af47-4552-bfe1-51af74d28095 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.154749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.155551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.155698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.156129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] PATH_INFO: `/v3/projects/3711d104eace46e4a7cf049602b2929b/users/a50176c885f844d4bb2469c64ca2dc49/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.172066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3711d104eace46e4a7cf049602b2929b, user_id=a50176c885f844d4bb2469c64ca2dc49, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.174804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.190306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 909/1805] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:58 2022] PUT /identity/v3/projects/3711d104eace46e4a7cf049602b2929b/users/a50176c885f844d4bb2469c64ca2dc49/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:58.201013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.201329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.201834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.230918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] MFA Rules not processed for user `a50176c885f844d4bb2469c64ca2dc49`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:58.232175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:58.302504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 897/1806] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:58.307483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.380784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f11d1f-abdc-400b-9576-e48116ae663d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.386410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.387052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.387334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.387606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.388285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.390287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.421370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 910/1807] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:43:58 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:58.425760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.436784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec10463-d418-434e-bf78-25bba1927e48 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.440632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:58.507586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.508590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.509032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.509451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.510353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.512501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.542716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 898/1808] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:43:58 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:43:58.546702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.559429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fb42-77be-4f0b-8ad9-5c478e204f89 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.565403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.566111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.566439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.566716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.567435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.570054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.590237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 911/1809] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:43:58.594886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.608746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97396f6b-bed0-467d-9dbb-5bfc63631834 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.614211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.615239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.615580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.615920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] PATH_INFO: `/v3/projects/dcec4e86df1045cba3676a047437e367/users/3ba1c04524ca46ec8bb814432e89c78c/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.644902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcec4e86df1045cba3676a047437e367, user_id=3ba1c04524ca46ec8bb814432e89c78c, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.645748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:58.686705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 899/1810] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:58 2022] PUT /identity/v3/projects/dcec4e86df1045cba3676a047437e367/users/3ba1c04524ca46ec8bb814432e89c78c/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:58.689152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:58.726190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839862d-4138-498e-9c42-0567a6e0b75f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:58.936757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:58.938410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:58.938772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:58.939278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:58.943997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:58.950066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8106f7c-64b9-4a9b-9cac-18a6c09927c8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:59.012308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 912/1811] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:43:59.015918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:59.046854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c175-f426-4a88-a9d7-23b72592496e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:59.071276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:59.073479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:59.074261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:59.076031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] PATH_INFO: `/v3/projects/dcec4e86df1045cba3676a047437e367/users/3ba1c04524ca46ec8bb814432e89c78c/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:59.085126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcec4e86df1045cba3676a047437e367, user_id=3ba1c04524ca46ec8bb814432e89c78c, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:59.102641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:59.137978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 900/1812] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:43:59 2022] PUT /identity/v3/projects/dcec4e86df1045cba3676a047437e367/users/3ba1c04524ca46ec8bb814432e89c78c/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:43:59.158282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:59.158812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:59.159278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:59.227451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] MFA Rules not processed for user `3ba1c04524ca46ec8bb814432e89c78c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:43:59.229710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:59.239112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:43:59.414862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 913/1813] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:43:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:43:59.430708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:43:59.433429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:43:59.645867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77164b5-e28d-44a0-8457-a49bf9b287bf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:43:59.655349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:43:59.656168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:43:59.657949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:43:59.658681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:43:59.688779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:43:59.691441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28f30-182d-409c-99ba-df0e30e6a6ac service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:43:59.717792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 901/1814] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:43:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:43:59.979231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:00.229073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7be50-df3b-428f-9e39-f3382f5fb523 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:00.243325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:00.246816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:00.248280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:00.250084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] PATH_INFO: `/v3/users/a50176c885f844d4bb2469c64ca2dc49` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:00.252153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50176c885f844d4bb2469c64ca2dc49)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:00.254751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:00.396164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50176c885f844d4bb2469c64ca2dc49'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:00.415953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50176c885f844d4bb2469c64ca2dc49'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:00.417062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50176c885f844d4bb2469c64ca2dc49'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:00.417886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50176c885f844d4bb2469c64ca2dc49'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:00.481787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50176c885f844d4bb2469c64ca2dc49'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:00.532405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 914/1815] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:43:59 2022] DELETE /identity/v3/users/a50176c885f844d4bb2469c64ca2dc49 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:00.544698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:00.544698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:00.544698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:00.574178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:00.575936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:00.675035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:00.676630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:00.751657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:00.754546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 902/1816] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:00.802340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:00.803956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:00.813340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:00.934372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a8edf-32bc-4192-a7bd-ca6431bcf050 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:00.952865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:00.953857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:00.954141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:00.954491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:00.971761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:00.971761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9901-aebe-4083-8b54-a08d4337d9e2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:00.982698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 903/1817] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:00.983534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bd9b1-7949-4908-8d8b-41613745b84d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:00.994348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:00.995715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:00.996456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:00.997223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] PATH_INFO: `/v3/users/3fe0b65eff7b4fb58e642c19ac786b97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:01.005109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe0b65eff7b4fb58e642c19ac786b97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:01.005396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:01.080382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe0b65eff7b4fb58e642c19ac786b97'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.097391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe0b65eff7b4fb58e642c19ac786b97'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.104340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe0b65eff7b4fb58e642c19ac786b97'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.105161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe0b65eff7b4fb58e642c19ac786b97'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.134610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe0b65eff7b4fb58e642c19ac786b97'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.161633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 915/1818] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:00 2022] DELETE /identity/v3/users/3fe0b65eff7b4fb58e642c19ac786b97 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:01.165313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:01.165520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:01.165684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:01.243042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:01.246001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:01.419461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 904/1819] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:01.433975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:01.436540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:01.570972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4e7fc-2baa-4ed7-b7ef-4285b5d5c5f8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:01.579143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:01.580194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:01.581040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:01.581496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:01.617771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:01.620802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:01.637238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 916/1820] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:01.822373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:01.825822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:01.849445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5224a-f97a-4bb1-9291-c2dc33a16eab None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:01.859032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:01.860799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:01.861385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:01.861980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] PATH_INFO: `/v3/projects/3711d104eace46e4a7cf049602b2929b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:01.864481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3711d104eace46e4a7cf049602b2929b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:01.867890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:01.946381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3711d104eace46e4a7cf049602b2929b', 'request_id': 'req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.961136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3711d104eace46e4a7cf049602b2929b', 'request_id': 'req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:01.971537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3711d104eace46e4a7cf049602b2929b', 'request_id': 'req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.002433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] The token cache is being invalidate because project 3711d104eace46e4a7cf049602b2929b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:02.002769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.003778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.006356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 905/1821] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:01 2022] DELETE /identity/v3/projects/3711d104eace46e4a7cf049602b2929b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:02.022422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:02.045076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fece9a-a604-4463-8ff2-2fd59e0496cf service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:02.122915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:02.142508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:02.143628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:02.144103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:02.144574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] PATH_INFO: `/v3/users/3ba1c04524ca46ec8bb814432e89c78c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:02.147538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba1c04524ca46ec8bb814432e89c78c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:02.154297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:02.289592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba1c04524ca46ec8bb814432e89c78c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.303567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba1c04524ca46ec8bb814432e89c78c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.304919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba1c04524ca46ec8bb814432e89c78c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.306137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba1c04524ca46ec8bb814432e89c78c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.332165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba1c04524ca46ec8bb814432e89c78c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.370158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 917/1822] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:02 2022] DELETE /identity/v3/users/3ba1c04524ca46ec8bb814432e89c78c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:02.518417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:02.531975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:02.570407 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5d574-3ea9-4576-85cb-088b5ca1b40f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:02.647335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:02.747055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:02.748004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:02.765844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:02.766294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] PATH_INFO: `/v3/projects/c3f478bfe3b34203bcb7a75cfc134563` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:02.769248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f478bfe3b34203bcb7a75cfc134563)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:02.771413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:02.889168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f478bfe3b34203bcb7a75cfc134563', 'request_id': 'req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.914279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f478bfe3b34203bcb7a75cfc134563', 'request_id': 'req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.929929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f478bfe3b34203bcb7a75cfc134563', 'request_id': 'req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.974165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] The token cache is being invalidate because project c3f478bfe3b34203bcb7a75cfc134563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:02.974720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.977122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:02.983256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 906/1823] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:02 2022] DELETE /identity/v3/projects/c3f478bfe3b34203bcb7a75cfc134563 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:02.987766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:02.991079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:03.031759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20c80f-8c49-4b0c-ba7d-aac5b485324b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:03.170511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:03.171980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:03.172330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:03.172803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] PATH_INFO: `/v3/users/8d1ebf53126446d49920625730e1c9ea` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:03.175770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1ebf53126446d49920625730e1c9ea)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:03.181845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:03.183977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:03.185988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:03.224495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9df1ce-94a7-4eee-a5b5-67429e32e9a4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:03.244649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:03.245700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:03.246161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:03.246597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] PATH_INFO: `/v3/users/177eadcf11b44bb385e3e21824ae324a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:03.265957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=177eadcf11b44bb385e3e21824ae324a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:03.265957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:03.276726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1ebf53126446d49920625730e1c9ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.303982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1ebf53126446d49920625730e1c9ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.305478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1ebf53126446d49920625730e1c9ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.305478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1ebf53126446d49920625730e1c9ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.337114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:03.371821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1ebf53126446d49920625730e1c9ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.411185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 918/1824] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:02 2022] DELETE /identity/v3/users/8d1ebf53126446d49920625730e1c9ea => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:03.412270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:03.417162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177eadcf11b44bb385e3e21824ae324a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.417460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:03.419062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177eadcf11b44bb385e3e21824ae324a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.444588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177eadcf11b44bb385e3e21824ae324a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.467370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177eadcf11b44bb385e3e21824ae324a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.468692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f8811fb6-f944-4559-ad15-479a49340619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177eadcf11b44bb385e3e21824ae324a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.538598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 907/1825] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:03 2022] DELETE /identity/v3/users/177eadcf11b44bb385e3e21824ae324a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:03.544949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:03.545893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:03.546859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:03.571643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02badd-e2ba-4233-b788-8b10a944a695 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:03.611241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:03.612938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:03.678750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:03.698652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:03.700704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:03.701466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:03.701936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] PATH_INFO: `/v3/projects/dcec4e86df1045cba3676a047437e367` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:03.703968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcec4e86df1045cba3676a047437e367)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:03.706186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:03.757323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 908/1826] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:03.781963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:03.783409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:03.889688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcec4e86df1045cba3676a047437e367', 'request_id': 'req-676903f3-4523-47fa-b6ed-1d002075b852'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.915424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcec4e86df1045cba3676a047437e367', 'request_id': 'req-676903f3-4523-47fa-b6ed-1d002075b852'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.939998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcec4e86df1045cba3676a047437e367', 'request_id': 'req-676903f3-4523-47fa-b6ed-1d002075b852'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.964973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1e79-9d6e-4b76-a4e4-9050e7e46860 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:03.988257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] The token cache is being invalidate because project dcec4e86df1045cba3676a047437e367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:03.988795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:03.993109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-676903f3-4523-47fa-b6ed-1d002075b852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.007049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 919/1827] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:03 2022] DELETE /identity/v3/projects/dcec4e86df1045cba3676a047437e367 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:04.011232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.011776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.012184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.086887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:04.088499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.093109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.094452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.095065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.095542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.101692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.112265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:04.294615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 920/1828] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:04.295585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.298168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.324404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.342506 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d641295-b948-4c4b-bea1-a050a59fa662 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.342907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f38dfc-7c75-4386-bd07-0b15e2a0bdc9 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.365605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.366617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.367267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.367420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] PATH_INFO: `/v3/projects/4aa73d9a8d184a47b5981476e6c52be0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.385614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa73d9a8d184a47b5981476e6c52be0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.385614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.385948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 909/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:04.385948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.385948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.385948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.413517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:04.415618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.434255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 910/1830] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:04.440930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.463018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa73d9a8d184a47b5981476e6c52be0', 'request_id': 'req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.467948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b2a1-fbfc-4863-af09-8cd42c936889 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.475238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa73d9a8d184a47b5981476e6c52be0', 'request_id': 'req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.487049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa73d9a8d184a47b5981476e6c52be0', 'request_id': 'req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.504627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] The token cache is being invalidate because project 4aa73d9a8d184a47b5981476e6c52be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:04.504959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.505798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ed9ae47c-8e1c-4c8b-bfab-8ea3a03ac263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:04.508719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 921/1831] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:04 2022] DELETE /identity/v3/projects/4aa73d9a8d184a47b5981476e6c52be0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:04.517684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3d6f-7eb8-4175-bc63-bec16e07cbfb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.518187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3d6f-7eb8-4175-bc63-bec16e07cbfb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.518751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e3d6f-7eb8-4175-bc63-bec16e07cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.539435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:04.564141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3d1e3d6f-7eb8-4175-bc63-bec16e07cbfb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:04.566604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3d1e3d6f-7eb8-4175-bc63-bec16e07cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.642168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.643247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.644137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.644257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.645580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:04.646143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.649145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.677745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 911/1832] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:04.680474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.685738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.721542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36afd-be7b-43cf-b5f4-4b3b60a65d61 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.726587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 922/1833] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:04.732117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.733004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.733549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.734326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.735462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.736039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.737351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.739286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.740991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.794744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:04.798282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:04.809688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 912/1834] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:04.816998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.888477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:04.905101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b5b9a-f49e-4452-8322-8245b755d0e3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.911205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.911979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.912725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.913004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.913748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.915751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.926953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 923/1835] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:04.931837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.944891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 913/1836] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:04.950881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:04.953297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007635-044d-4965-ac82-ef721d1b325a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.965813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.966746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.967183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.967493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.968483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cca679-30e9-4f69-aed5-06c36d00e447 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:04.968970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:04.969765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.973043 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.974917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:04.976033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:04.977963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:04.977963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:04.977963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:04.980037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:04.996931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 914/1837] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:04.997127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 924/1838] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:44:05.000024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.001027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.022524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35f5d-6a60-42d1-a728-f0d94e37d72a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.028382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.029096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.029380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.029839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.030195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473af5b7-ad15-49fc-809f-193368cb5e8a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.030575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.032806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.037195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.037876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.038155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.038421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.039393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.041045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.065190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 915/1839] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:05.066456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.078015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 925/1840] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:04 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:05.078438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9ec47-1723-4d54-ad8e-98d09091aaaf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.081582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.161846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da30f4-58ff-4c5c-81e0-29c754e302ca admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.167891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.168575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.168864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.169211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.169857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.170091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.170892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.172103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.172103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] PATH_INFO: `/v3/projects/d67a0073f49e430d97ad3fc692aaf6d5/users/379e3f00c2134077b83cc46932758409/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.172410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.196137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67a0073f49e430d97ad3fc692aaf6d5, user_id=379e3f00c2134077b83cc46932758409, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.199366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3593a-408b-4f67-ab30-a34a661fdc04 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.202086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:05.204112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 926/1841] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:05 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:05.209530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.222994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 916/1842] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:05 2022] PUT /identity/v3/projects/d67a0073f49e430d97ad3fc692aaf6d5/users/379e3f00c2134077b83cc46932758409/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:05.240464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2166-1afb-4efa-91ef-e87a3f832669 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.241645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.242192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.242713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.312706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] MFA Rules not processed for user `379e3f00c2134077b83cc46932758409`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:05.315045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.353517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.354958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.354958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.355136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.355939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.358205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.375675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 927/1843] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:05.377099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.379859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.402957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54198e1-15e6-4cad-95e6-e40c5dae064e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.412501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.414218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.415880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.416490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] PATH_INFO: `/v3/projects/78079e342d0d4c1e8c9b267ab68a515e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.419519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78079e342d0d4c1e8c9b267ab68a515e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.420843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 917/1844] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:05.425610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.425715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.445215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4bce0-8d39-4675-914b-4f2772b5a1a5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.451386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.452144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.452437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.452714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.453518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.455187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.470635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 918/1845] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:05.470635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.470635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.494133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:05.517733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78079e342d0d4c1e8c9b267ab68a515e', 'request_id': 'req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:05.529157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78079e342d0d4c1e8c9b267ab68a515e', 'request_id': 'req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:05.540420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78079e342d0d4c1e8c9b267ab68a515e', 'request_id': 'req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:05.564242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] The token cache is being invalidate because project 78079e342d0d4c1e8c9b267ab68a515e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:05.564547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:05.565473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:05.570042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 928/1846] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:05 2022] DELETE /identity/v3/projects/78079e342d0d4c1e8c9b267ab68a515e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:05.574363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.578416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.579285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6270-6833-474e-8b68-82d4a9b84f9f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.687148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.687806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.688081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.688504 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.699393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.723021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5f8a-1abf-41a4-9d17-d6c96845c4f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.737105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.737755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.738159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.738442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] PATH_INFO: `/v3/projects/93e1be609d034c03a20accb8ac555a23/users/f0a9e432736040c78d3b2a492d211813/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.773648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e1be609d034c03a20accb8ac555a23, user_id=f0a9e432736040c78d3b2a492d211813, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.783386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.786238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:05.822337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 929/1847] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:05 2022] PUT /identity/v3/projects/93e1be609d034c03a20accb8ac555a23/users/f0a9e432736040c78d3b2a492d211813/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:05.825339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:05.827614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.832502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.836133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca05f6e-b25a-4324-b0e5-a7b2dbffe552 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:05.858265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 919/1848] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:05.861056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.861674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.862231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.915889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] MFA Rules not processed for user `f0a9e432736040c78d3b2a492d211813`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:05.918507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:05.944398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cae41-fcc3-4335-8aa1-c832776a6271 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:05.953277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:05.954171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:05.954472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:05.954823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] PATH_INFO: `/v3/users/6f58776839594e7d9ab7f58099f7aacf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:05.956154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f58776839594e7d9ab7f58099f7aacf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:05.958163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.031848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:06.035293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f58776839594e7d9ab7f58099f7aacf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.046109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f58776839594e7d9ab7f58099f7aacf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.047165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f58776839594e7d9ab7f58099f7aacf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.048209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f58776839594e7d9ab7f58099f7aacf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.054278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 920/1849] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:06.058798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.059677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.066547 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f58776839594e7d9ab7f58099f7aacf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.077283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e312d-7a70-491f-aa88-f03e99e9785a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.083733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.084400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.084697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.084954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] PATH_INFO: `/v3/users/379e3f00c2134077b83cc46932758409` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.086219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 930/1850] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:05 2022] DELETE /identity/v3/users/6f58776839594e7d9ab7f58099f7aacf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:06.086663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379e3f00c2134077b83cc46932758409)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.087667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.089947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.090805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.173592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379e3f00c2134077b83cc46932758409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.174491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379e3f00c2134077b83cc46932758409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.187519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379e3f00c2134077b83cc46932758409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.198611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379e3f00c2134077b83cc46932758409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.200397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c7d51a7-9676-408b-a2ce-f7fcf1704af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379e3f00c2134077b83cc46932758409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.224502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd14578-b5a4-47cd-ad9d-0d33c0b5e011 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.237644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.238799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.239270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.239725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.240926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.244607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.250015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 921/1851] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:06 2022] DELETE /identity/v3/users/379e3f00c2134077b83cc46932758409 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:06.252548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.253327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.253857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.285058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:06.287478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.312363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 931/1852] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:06.318206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.320370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.371751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:06.414588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 922/1853] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:06.426928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501f83a-6517-4612-b5fe-03ed390661fd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.430392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.432629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.433516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.434519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.434830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.435494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.435993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.438333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.467026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 932/1854] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:06.472557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.489908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b732a-1ce8-499d-ac54-332fafb1665a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.499641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.499641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.499641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.500080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.501361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.504309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.527143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 933/1855] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:06.527143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.527143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.541967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54aebc-3c15-4dfe-b62c-d2f3a852825c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.545660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cfd85-9312-4c83-a488-4be108c6001c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.548065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.548843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.549296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.550015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] PATH_INFO: `/v3/projects/3229c75f14f74a38a2bff23cec529699` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.551484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3229c75f14f74a38a2bff23cec529699)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.553843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.580478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:06.618127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.618889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.619101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.619340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.624690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3229c75f14f74a38a2bff23cec529699', 'request_id': 'req-9c51b769-ecdd-4272-b691-f649e111ef7a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.629548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.631138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.635694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3229c75f14f74a38a2bff23cec529699', 'request_id': 'req-9c51b769-ecdd-4272-b691-f649e111ef7a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.642772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 923/1856] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:06.643719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.644268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3229c75f14f74a38a2bff23cec529699', 'request_id': 'req-9c51b769-ecdd-4272-b691-f649e111ef7a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.654186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e19f61-a0ef-44af-b805-a52c5922e6c9 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.667714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] The token cache is being invalidate because project 3229c75f14f74a38a2bff23cec529699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:06.669381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.670641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c51b769-ecdd-4272-b691-f649e111ef7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:06.674067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 934/1857] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:06 2022] DELETE /identity/v3/projects/3229c75f14f74a38a2bff23cec529699 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:06.681913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab4f13-104f-4070-9735-29aa4675a5b9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.682472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab4f13-104f-4070-9735-29aa4675a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.682956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab4f13-104f-4070-9735-29aa4675a5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.719015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-88ab4f13-104f-4070-9735-29aa4675a5b9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:06.721458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88ab4f13-104f-4070-9735-29aa4675a5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.734627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.735684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.736157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.736543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] PATH_INFO: `/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba/users/2ebf2ba5e8cd47cca8cedafa8044196a/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.758081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfd93183ffd415ca3ebd3c3346d3fba, user_id=2ebf2ba5e8cd47cca8cedafa8044196a, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.761489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.780261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 924/1858] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:06 2022] PUT /identity/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba/users/2ebf2ba5e8cd47cca8cedafa8044196a/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:06.784956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.786953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.833400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:06.881747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 935/1859] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:06.890131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.890507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.892331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.910350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931949b-6360-4135-8e0c-c71a1fc1abec admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.918855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.920054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.920625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.921180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.922632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.924215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:06.926324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:06.926455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.936327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 936/1860] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:06.940789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.941647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 925/1861] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:06.948402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.951397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf6964-4a58-40fc-a14d-7ddb5a98fd2d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.956368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.957028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.957308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.957604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.958704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:06.958704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.960279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.967035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eafe7-9e55-4e61-81e6-149a190b7ecc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:06.974188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:06.975475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:06.975900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:06.976331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] PATH_INFO: `/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba/users/2ebf2ba5e8cd47cca8cedafa8044196a/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:06.978650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfd93183ffd415ca3ebd3c3346d3fba, user_id=2ebf2ba5e8cd47cca8cedafa8044196a, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:06.980151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 937/1862] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:44:06.981144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:06.981898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96f4a2-4ac9-4c10-9b95-eb41e2f7433f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:06.993150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1fff-1dd4-45ec-9c61-720a3289246f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.003340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 926/1863] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:06 2022] PUT /identity/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba/users/2ebf2ba5e8cd47cca8cedafa8044196a/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:07.018376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.019101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.019101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.086226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] MFA Rules not processed for user `2ebf2ba5e8cd47cca8cedafa8044196a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:07.088101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.119010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.119808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.120180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.120731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.121538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.123914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.161065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 938/1864] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:06 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:07.162115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.163882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.187088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:07.259581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 927/1865] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:07.260749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.283488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf937f-eded-4e60-9ec4-5620361ad102 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.289748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.290490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.290837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.291371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.292147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.295003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.299303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d10e6-ca68-4c06-9b60-6b9f23d4f20a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.306162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.307408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.308005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.308576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] PATH_INFO: `/v3/projects/d67a0073f49e430d97ad3fc692aaf6d5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.310920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67a0073f49e430d97ad3fc692aaf6d5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.313491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.338097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 928/1866] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/users => generated 282 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:07.338097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.340657 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.401370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67a0073f49e430d97ad3fc692aaf6d5', 'request_id': 'req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:07.430153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67a0073f49e430d97ad3fc692aaf6d5', 'request_id': 'req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:07.437003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67a0073f49e430d97ad3fc692aaf6d5', 'request_id': 'req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:07.447637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842cbfe-82ee-49cf-bf9e-68de5030591a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.452921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] The token cache is being invalidate because project d67a0073f49e430d97ad3fc692aaf6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:07.453317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:07.454082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:07.459407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 939/1867] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:07 2022] DELETE /identity/v3/projects/d67a0073f49e430d97ad3fc692aaf6d5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:07.459407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.459407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.485857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:07.509009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.509549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.509852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.510256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.512129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.539298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14365-64d3-49f1-a2ec-869cb45d6fa6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.544405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.545324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.545324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.545681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.546316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.548003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac051-e43d-4861-ad46-b503dc5983b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.561782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 940/1868] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:07.561782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.561782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.561782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.584090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:07.585353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.586046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.587492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d26958-34f9-4c90-8bc9-6ec311bca52b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.594341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 941/1869] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:07.601725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.602725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 929/1870] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:07.606424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.606782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.607123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.612868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1c56-4bdc-4cca-826a-da247dbab0c7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.619021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.619707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.619986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.620265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.621165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.622631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.631438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:07.632649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.634750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 942/1871] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:07.639400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.641214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 930/1872] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:07.647291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.650717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf312-5ec6-439d-adc9-3f6189f113b6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.655656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.656353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.656629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.656898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] PATH_INFO: `/v3/projects/c7edc08709984fa2b5a9bc2fd668123f/users/5be2dd500e9d40b18869401f8669a2ba/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.657467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c45a-4d1e-4c14-9089-9ec2efa6efa5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.663238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.664516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.664810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.665085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.665854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:07.666172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.667896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.676042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7edc08709984fa2b5a9bc2fd668123f, user_id=5be2dd500e9d40b18869401f8669a2ba, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.678773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde94d0-ad47-41c0-884d-73d8b2e76b9d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.684949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 931/1873] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:07.690088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.690280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:07.697808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 943/1874] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:07 2022] PUT /identity/v3/projects/c7edc08709984fa2b5a9bc2fd668123f/users/5be2dd500e9d40b18869401f8669a2ba/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:07.705703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda465fa-dc4c-42ba-a47c-5071b471736b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.707581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.707874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.708178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.740025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] MFA Rules not processed for user `5be2dd500e9d40b18869401f8669a2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:07.741279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.765597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.766125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.766404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.766910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.767344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.769422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.800515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 932/1875] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:07.801690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.803372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.825823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 944/1876] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:07.826749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.846576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa2ed5-7d51-47d5-8a6e-c20ef800660f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.876270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:07.880651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64fd01-d4c2-4115-8750-1f9d76f5f375 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.893740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.894455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.894784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.894988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.897027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:07.909290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.909966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.910241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.910536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.911205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.913267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.937065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 945/1877] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:07.937952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.949078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104408f7-da5c-4798-b4ab-f21c83eaa700 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.953803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:07.954496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:07.954704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:07.954986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:07.955642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.957578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.961505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:07.963125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:07.973425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 933/1878] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:07.974356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:07.987011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1ab5c-dea7-4dc3-abf2-e6477f451a4f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:07.989325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 946/1879] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:07.993938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.064033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.064773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.065053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.065321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.065909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:08.066232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.068128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.075013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfcbe-520a-4821-a3b8-8f241bf86232 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.079277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 934/1880] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:08.085191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.087226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.088042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.088567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.088969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.090210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.093282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.100403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb0f9-b2f3-4f16-9bdf-f0f1447d06f6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.110194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.110855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.111143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.111419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.112211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.114309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.131418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 947/1881] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:07 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:08.133106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.137211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 935/1882] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:08.146170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.153352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5454-7dc7-4a8f-9c8a-d50c42939fa9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.165102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.165102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.167479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.167479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.167844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37fa60-0249-4b7d-88f4-71bab9f4521e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.168254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.171402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.184189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.185656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.186158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.186642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] PATH_INFO: `/v3/projects/7908dd73c39a4d30ac4b2bd9c69002f8/users/c8d88d538a42449292fb5f01da24dd64/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.198804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 948/1883] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:08.205723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.208653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7908dd73c39a4d30ac4b2bd9c69002f8, user_id=c8d88d538a42449292fb5f01da24dd64, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.211908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19896273-aea6-4856-9ca5-1b6a7adc9c9c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.225744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4dee0-cae4-479c-af0e-773d40d6d2bb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.234202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 936/1884] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:08 2022] PUT /identity/v3/projects/7908dd73c39a4d30ac4b2bd9c69002f8/users/c8d88d538a42449292fb5f01da24dd64/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:08.252966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.253716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.254365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.310949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] MFA Rules not processed for user `c8d88d538a42449292fb5f01da24dd64`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:08.313410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:08.336568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.337694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.338504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.338730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] PATH_INFO: `/v3/projects/1f27e84a0bb0469aaef2384b150abe92/users/7ce0c31e20814781b24d6eb414435912/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.371468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f27e84a0bb0469aaef2384b150abe92, user_id=7ce0c31e20814781b24d6eb414435912, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.378108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.413063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 949/1885] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:08 2022] PUT /identity/v3/projects/1f27e84a0bb0469aaef2384b150abe92/users/7ce0c31e20814781b24d6eb414435912/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:08.416022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.489599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:08.515104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 937/1886] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:08.533493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.536656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:08.589927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8038455-9ee3-47ba-be89-30496a9e5c3d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.600196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.601519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.602140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.602689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.604262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.608568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.646600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 950/1887] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:08.650665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:08.670964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29526037-ef0d-4c5a-9430-ec29bc468228 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.684400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.685593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.686184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.686572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] PATH_INFO: `/v3/projects/1f27e84a0bb0469aaef2384b150abe92/users/7ce0c31e20814781b24d6eb414435912/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.689599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f27e84a0bb0469aaef2384b150abe92, user_id=7ce0c31e20814781b24d6eb414435912, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.694775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084efc3-e714-4e32-be67-c6c88206a2a7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.724946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 951/1888] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:08 2022] PUT /identity/v3/projects/1f27e84a0bb0469aaef2384b150abe92/users/7ce0c31e20814781b24d6eb414435912/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:08.729490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.729973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.730349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.730672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0c9ba-a86b-41e8-aa0d-788ffdb3ae79 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:08.781535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] MFA Rules not processed for user `7ce0c31e20814781b24d6eb414435912`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:08.782843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:08.818059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:08.818744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:08.819272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:08.819986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:08.834583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:08.898329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:08.900341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:08.913817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 952/1889] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:08.929371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 938/1890] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:08.937599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:09.048833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ade7-c36d-420f-94e8-1ae127746e9a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:09.056503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:09.058382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:09.058902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:09.059333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:09.078681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:09.082222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:09.093745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 953/1891] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:08 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:09.295972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:09.305431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da820-b420-4301-a505-7f2793ef2e23 service glance] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:09.309582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:09.310241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:09.310518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:09.310778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:09.363565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:09.386793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:09.388251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdd9d0-7d6d-4aea-ba02-7080982df369 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:09.398651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 939/1892] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:09 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:12.493106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:12.658000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d4bd5-2587-4cc8-b6b0-9fcb2ac60690 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:12.666515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:12.667618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:12.668103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:12.668583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] PATH_INFO: `/v3/users/5be2dd500e9d40b18869401f8669a2ba` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:12.671059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be2dd500e9d40b18869401f8669a2ba)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:12.674588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:12.860522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be2dd500e9d40b18869401f8669a2ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:12.890935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be2dd500e9d40b18869401f8669a2ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:12.893777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be2dd500e9d40b18869401f8669a2ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:12.895922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be2dd500e9d40b18869401f8669a2ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:12.943464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:12.947192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be2dd500e9d40b18869401f8669a2ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.024426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 954/1893] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:12 2022] DELETE /identity/v3/users/5be2dd500e9d40b18869401f8669a2ba => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:13.030316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:13.031859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:13.033589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:13.100016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:13.103088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:13.305689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 940/1894] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:13.320236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:13.323964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:13.450295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd7c3b-5258-4ae5-b597-23cc8446566f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:13.458489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:13.459548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:13.460585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:13.461077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:13.482833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:13.484208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687fca-d705-4704-ab06-be36a4601f9c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:13.500426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 955/1895] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:13.667349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:13.668960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:13.690804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6b3e6-12f3-4a6d-bf60-0e6ca379d9a0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:13.699114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:13.700128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:13.700628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:13.701120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] PATH_INFO: `/v3/users/26104e6d2a214fa69139e2bf90b53c74` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:13.702751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26104e6d2a214fa69139e2bf90b53c74)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:13.705366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:13.767276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26104e6d2a214fa69139e2bf90b53c74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.768189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26104e6d2a214fa69139e2bf90b53c74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.778191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26104e6d2a214fa69139e2bf90b53c74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.789950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26104e6d2a214fa69139e2bf90b53c74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.794613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26104e6d2a214fa69139e2bf90b53c74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:13.836975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 941/1896] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:13 2022] DELETE /identity/v3/users/26104e6d2a214fa69139e2bf90b53c74 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:13.838838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:13.839857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:13.840192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:13.902909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:13.905179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:14.117095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 956/1897] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:14.160609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:14.164778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:14.315260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1c96a-03eb-48e1-8a69-799edfcf1d0a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:14.323869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:14.325053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:14.325532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:14.326128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:14.349344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:14.351031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:14.351226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:14.355340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:14.365075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:14.372349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612db08-f4ee-43b8-a0fa-7c7667d7d319 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:14.387064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 942/1898] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:14.388994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:14.390902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:14.391265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:14.392133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] PATH_INFO: `/v3/projects/c7edc08709984fa2b5a9bc2fd668123f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:14.394332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7edc08709984fa2b5a9bc2fd668123f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:14.399418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:14.559346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7edc08709984fa2b5a9bc2fd668123f', 'request_id': 'req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.584227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7edc08709984fa2b5a9bc2fd668123f', 'request_id': 'req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.601688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7edc08709984fa2b5a9bc2fd668123f', 'request_id': 'req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.639814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] The token cache is being invalidate because project c7edc08709984fa2b5a9bc2fd668123f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:14.639814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.640789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.642767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 957/1899] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:14 2022] DELETE /identity/v3/projects/c7edc08709984fa2b5a9bc2fd668123f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:14.648991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:14.667154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3540a8-ac80-409e-b53e-2457748c00fc service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:14.774630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:14.775852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:14.776382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:14.776790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] PATH_INFO: `/v3/users/7ce0c31e20814781b24d6eb414435912` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:14.778816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce0c31e20814781b24d6eb414435912)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:14.781261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:14.930956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce0c31e20814781b24d6eb414435912'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.932298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce0c31e20814781b24d6eb414435912'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.967328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce0c31e20814781b24d6eb414435912'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.990013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce0c31e20814781b24d6eb414435912'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:14.997524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce0c31e20814781b24d6eb414435912'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:15.084041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 943/1900] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:14 2022] DELETE /identity/v3/users/7ce0c31e20814781b24d6eb414435912 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:15.758376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:15.761154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:15.785424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ebbb0-5398-4f90-bcd6-614c6e7a3c9b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:15.817985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:15.929451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:15.933912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:15.946161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:15.947316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:15.947839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:15.948326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] PATH_INFO: `/v3/users/f0a9e432736040c78d3b2a492d211813` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:15.953352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a9e432736040c78d3b2a492d211813)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:15.970015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:15.979589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bc619-e93b-40cf-a1bc-cf7dbd21442b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:15.990313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:15.991986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:15.992606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:15.993316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] PATH_INFO: `/v3/projects/1f27e84a0bb0469aaef2384b150abe92` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:15.996418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f27e84a0bb0469aaef2384b150abe92)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:16.003097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:16.097864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a9e432736040c78d3b2a492d211813'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.117060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a9e432736040c78d3b2a492d211813'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.118211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a9e432736040c78d3b2a492d211813'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.119436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a9e432736040c78d3b2a492d211813'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.170994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2a319ad-4f52-4827-b678-27b27e9de7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a9e432736040c78d3b2a492d211813'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.187075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f27e84a0bb0469aaef2384b150abe92', 'request_id': 'req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.199893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 958/1901] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:15 2022] DELETE /identity/v3/users/f0a9e432736040c78d3b2a492d211813 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:16.202231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f27e84a0bb0469aaef2384b150abe92', 'request_id': 'req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.202454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:16.202799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:16.203116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:16.212156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f27e84a0bb0469aaef2384b150abe92', 'request_id': 'req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.248414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:16.264225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] The token cache is being invalidate because project 1f27e84a0bb0469aaef2384b150abe92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:16.271963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.274817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dd1904a3-bdb3-4184-bb7e-c61a745f95a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.284202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 944/1902] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:15 2022] DELETE /identity/v3/projects/1f27e84a0bb0469aaef2384b150abe92 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:16.286157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:16.289179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.292117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:16.292695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:16.293172 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:16.372367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:16.374554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.499246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 959/1903] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:16.529485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:16.532256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.617740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 945/1904] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:16.626285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:16.628491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.629716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:16.652875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1b66-d200-4153-bb31-d973045c88b0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:16.662001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:16.663331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:16.663979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:16.664464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] PATH_INFO: `/v3/projects/6e61cb6b37994196a0ef1b5910524917` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:16.668893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e61cb6b37994196a0ef1b5910524917)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:16.674013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:16.688313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0518c5-e8ca-46f5-bcd9-91a7981a24eb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:16.695313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:16.696617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:16.697117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:16.697730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:16.716500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:16.717933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa37aee-3d72-4565-b402-b37c59136f44 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:16.739336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 960/1905] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:16.752974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:16.755569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:16.770155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:16.815432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e61cb6b37994196a0ef1b5910524917', 'request_id': 'req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.832021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e61cb6b37994196a0ef1b5910524917', 'request_id': 'req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.834180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:16.837479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.857234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e61cb6b37994196a0ef1b5910524917', 'request_id': 'req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.902322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] The token cache is being invalidate because project 6e61cb6b37994196a0ef1b5910524917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:16.917779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.918733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:16.926650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 946/1906] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:16 2022] DELETE /identity/v3/projects/6e61cb6b37994196a0ef1b5910524917 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:16.927777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:16.932212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:16.990069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 961/1907] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:16.992596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.026885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7c7f-4e1a-44f8-878b-3f9c44a71c94 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.035302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.036416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.036988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.037394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.038423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:17.039864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.042242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c061bb-dc92-4f86-bf74-f52a9c6f6a63 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.043325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.049481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.050671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.050671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.051108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] PATH_INFO: `/v3/users/31c723d379b449a091eb41bb0226a50f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.052667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c723d379b449a091eb41bb0226a50f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.054978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.064795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 962/1908] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:17.070269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.083509 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e83c7-1d6c-4953-9b01-2eec06aab7ca admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.090617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.091325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.091548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.091821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.092513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.094642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.107793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:17.163735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c723d379b449a091eb41bb0226a50f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:17.166276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c723d379b449a091eb41bb0226a50f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:17.181633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 963/1909] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:17 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:17.183311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.184007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c723d379b449a091eb41bb0226a50f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:17.194202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:17.198775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c723d379b449a091eb41bb0226a50f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:17.202538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c723d379b449a091eb41bb0226a50f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:17.238822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 947/1910] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:16 2022] DELETE /identity/v3/users/31c723d379b449a091eb41bb0226a50f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:17.325948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544807-46e1-41a0-8daf-ca18234aa3e0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.468508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:17.505230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.507191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.507974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.508806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.511231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.516014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.611129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 964/1911] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:17 2022] POST /identity/v3/users => generated 291 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:17.633013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.665049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccba97-133e-451d-9401-074c57e85eb1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.681743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.682837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.683191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.683643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.684981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.687664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.718217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 948/1912] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:17.722907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.741088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f859b-9466-45ad-9d7c-91fd6a257fb0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.758937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.759828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.762928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.763480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.767304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.770159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.784246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 965/1913] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:17.789166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.809547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34530142-8137-42f4-ad1a-f4f30e8d6cf7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:17.819005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:17.820234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:17.820944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:17.821179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] PATH_INFO: `/v3/projects/56228c9407654b649d1133df5553adf3/users/7a0763c596354ca083a3ced4aa274f64/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:17.855725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56228c9407654b649d1133df5553adf3, user_id=7a0763c596354ca083a3ced4aa274f64, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:17.862429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:17.912676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 949/1914] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:17 2022] PUT /identity/v3/projects/56228c9407654b649d1133df5553adf3/users/7a0763c596354ca083a3ced4aa274f64/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:17.913130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.913862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:17.916414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:17.941009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:17.958764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8a27-d66f-41ff-a656-29cbda05a353 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:18.091251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf660da-58c4-4477-a63b-e4ac6042c165 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:18.101289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:18.102859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.104241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.106021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] PATH_INFO: `/v3/projects/93e1be609d034c03a20accb8ac555a23` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.108986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e1be609d034c03a20accb8ac555a23)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:18.112814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:18.118891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:18.120045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.120624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.122426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] PATH_INFO: `/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.123219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4032e865e34f8ebdad024fc80fa1c0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:18.125648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:18.228448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4032e865e34f8ebdad024fc80fa1c0', 'request_id': 'req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.232494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e1be609d034c03a20accb8ac555a23', 'request_id': 'req-ce6eb365-242f-4bb9-9786-8d437b4f4e63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.244512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4032e865e34f8ebdad024fc80fa1c0', 'request_id': 'req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.256560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4032e865e34f8ebdad024fc80fa1c0', 'request_id': 'req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.258729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e1be609d034c03a20accb8ac555a23', 'request_id': 'req-ce6eb365-242f-4bb9-9786-8d437b4f4e63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.270972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e1be609d034c03a20accb8ac555a23', 'request_id': 'req-ce6eb365-242f-4bb9-9786-8d437b4f4e63'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.279186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] The token cache is being invalidate because project 7d4032e865e34f8ebdad024fc80fa1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:18.279691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.281713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a4ea397-8b32-417c-a5b5-42f381e8b6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.289471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 966/1915] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:17 2022] DELETE /identity/v3/projects/7d4032e865e34f8ebdad024fc80fa1c0 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:18.289471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501f57-11e6-4526-a32a-1e8801de133e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.289471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501f57-11e6-4526-a32a-1e8801de133e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.289471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501f57-11e6-4526-a32a-1e8801de133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.294944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:18.303252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] The token cache is being invalidate because project 93e1be609d034c03a20accb8ac555a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:18.303874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.305620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ce6eb365-242f-4bb9-9786-8d437b4f4e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.308385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 950/1916] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:17 2022] DELETE /identity/v3/projects/93e1be609d034c03a20accb8ac555a23 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:18.310141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.310744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.311263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.346076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc501f57-11e6-4526-a32a-1e8801de133e None None] MFA Rules not processed for user `7a0763c596354ca083a3ced4aa274f64`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:18.348191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc501f57-11e6-4526-a32a-1e8801de133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.366377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:18.367869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.485867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:18.552059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 951/1917] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:18.552059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:18.554727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.572611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 967/1918] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:18.579454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.579709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.579986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.584394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca070-2481-4e41-9931-a3c17ec2ba03 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:18.596385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:18.596385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.596839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.596839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] PATH_INFO: `/v3/users/2ebf2ba5e8cd47cca8cedafa8044196a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.598305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebf2ba5e8cd47cca8cedafa8044196a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:18.601744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:18.648340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:18.652433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.683710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 968/1919] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:18.694490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:18.699308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.725197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebf2ba5e8cd47cca8cedafa8044196a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.725992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebf2ba5e8cd47cca8cedafa8044196a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.733985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ca1f-f950-4a26-83c4-ff60d0ad4b46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:18.734445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebf2ba5e8cd47cca8cedafa8044196a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.743639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:18.745179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:18.745451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebf2ba5e8cd47cca8cedafa8044196a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.746343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebf2ba5e8cd47cca8cedafa8044196a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:18.746508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:18.747147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:18.748633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:18.753455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:18.773487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 952/1920] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:18 2022] DELETE /identity/v3/users/2ebf2ba5e8cd47cca8cedafa8044196a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:18.774827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:18.780594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.858684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 969/1921] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:18 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:18.864491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:18.868850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:18.892195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bafc5-a524-45e8-9fd1-62a482005c7d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:18.927458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:19.053332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.060255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.061007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.061879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.063030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0348de-7cd0-45ac-975a-02d0d6ca8dc5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.072109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18342848-224a-4452-a33a-57957707f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.073213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18342848-224a-4452-a33a-57957707f182 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.073704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18342848-224a-4452-a33a-57957707f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.074150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18342848-224a-4452-a33a-57957707f182 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.075232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18342848-224a-4452-a33a-57957707f182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.078967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:19.079902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.084986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.085896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18342848-224a-4452-a33a-57957707f182 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.132811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 953/1922] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:19.140086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.143261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 970/1923] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:18 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:19.148519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18342848-224a-4452-a33a-57957707f182 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.157947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba07c-8b29-498a-a667-62ff3d6cb62d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.179113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.186619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.187111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.187475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.188339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.190475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.192319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18342848-224a-4452-a33a-57957707f182 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.205154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.206747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.207497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.207965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.209213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.215257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.255310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 971/1924] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:19.256067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 954/1925] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:19 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:19.265849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.269838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.319438 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afec0-e7a3-45b1-87a1-f68346943d56 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.454197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cdd89-b328-4c7f-ab3b-3496a9a7b213 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.465136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.467184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.467184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.467184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.468733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.473663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.505396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.506928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.507307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.507760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] PATH_INFO: `/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b/users/3a43e158d19b4249b23585693cd983de/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.525133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 955/1926] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:19 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:19.526382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.544773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a9af4901c94b068ae46dd4f9fe2d5b, user_id=3a43e158d19b4249b23585693cd983de, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.546456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d2f9-0822-4a05-9925-9c37d80d76b6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.548759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.555726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.557143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.557424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.558539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.559980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.562177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.590732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 956/1927] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:19.592390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.599880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 972/1928] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:19 2022] PUT /identity/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b/users/3a43e158d19b4249b23585693cd983de/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:19.605226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.612721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97121-1434-489a-a693-c9895a82a02a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.632345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:19.712208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e07a-1b34-4a56-917f-91da1dd6fcc8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.720281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.721061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.721519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.721880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.723105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.725192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.731053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.732116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.732673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.733333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.734775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.740275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.742494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 973/1929] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:19.748751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.763911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 957/1930] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:19.766618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2d102-a9d5-4afe-9c6b-cc099223766a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.769202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.773029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:19.780700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.782244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.783376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.784171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] PATH_INFO: `/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b/users/3a43e158d19b4249b23585693cd983de/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.788790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a9af4901c94b068ae46dd4f9fe2d5b, user_id=3a43e158d19b4249b23585693cd983de, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.789447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:19.794735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f7fbc-56ac-45fd-85eb-bc833169e3b7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.796556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.802793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:19.803602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:19.803943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:19.804830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] PATH_INFO: `/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:19.807214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfd93183ffd415ca3ebd3c3346d3fba)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:19.809361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:19.826361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 974/1931] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:19 2022] PUT /identity/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b/users/3a43e158d19b4249b23585693cd983de/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:19.828411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:19.923864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfd93183ffd415ca3ebd3c3346d3fba', 'request_id': 'req-023262eb-0961-4db1-9162-1fc57767cdb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:19.948247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfd93183ffd415ca3ebd3c3346d3fba', 'request_id': 'req-023262eb-0961-4db1-9162-1fc57767cdb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:19.970163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a86c77-b008-49bb-aabd-b8a40aef7e19 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:19.971035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfd93183ffd415ca3ebd3c3346d3fba', 'request_id': 'req-023262eb-0961-4db1-9162-1fc57767cdb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:20.011507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] The token cache is being invalidate because project acfd93183ffd415ca3ebd3c3346d3fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:20.011991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:20.012748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-023262eb-0961-4db1-9162-1fc57767cdb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:20.014482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 958/1932] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:19 2022] DELETE /identity/v3/projects/acfd93183ffd415ca3ebd3c3346d3fba => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:20.019249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87fb-ff83-4508-a1f8-84b81ae6f0dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.019249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87fb-ff83-4508-a1f8-84b81ae6f0dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.019249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87fb-ff83-4508-a1f8-84b81ae6f0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.084116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d8fe87fb-ff83-4508-a1f8-84b81ae6f0dc None None] MFA Rules not processed for user `3a43e158d19b4249b23585693cd983de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:20.085848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d8fe87fb-ff83-4508-a1f8-84b81ae6f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.125530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.128397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.128897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.129171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] PATH_INFO: `/v3/projects/cdd7110c8e124d668a035bb88f7baf95/users/d0ab9fbdfbd54878ae227c661b1d5f89/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.156570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd7110c8e124d668a035bb88f7baf95, user_id=d0ab9fbdfbd54878ae227c661b1d5f89, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.162552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785e729-1b2b-417e-984e-d109b2118681 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.202489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 975/1933] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:19 2022] PUT /identity/v3/projects/cdd7110c8e124d668a035bb88f7baf95/users/d0ab9fbdfbd54878ae227c661b1d5f89/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:20.206760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.207444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.208022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.216053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 959/1934] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:20.217676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c212279-498f-461b-8410-a0d55439943a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.218317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c212279-498f-461b-8410-a0d55439943a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.218769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c212279-498f-461b-8410-a0d55439943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.246654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:20.248476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.270489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7c212279-498f-461b-8410-a0d55439943a None None] MFA Rules not processed for user `d0ab9fbdfbd54878ae227c661b1d5f89`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:20.272032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7c212279-498f-461b-8410-a0d55439943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.313287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:20.386504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 976/1935] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:20.388179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.390785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.436629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 960/1936] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:20.439034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.440979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.440979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.455583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:20.483025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:20.485463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.499925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a9dc6-fb7b-4c15-afe2-b3e34619933c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:20.507097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.508582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.510546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 961/1937] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:20.511003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.511397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.512732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.514821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:20.559905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cc560-ed8c-420f-b4b2-de9677afe524 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:20.569286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.570510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.571008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.571493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.572876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.578004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.630113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 962/1938] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:20.631038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.636639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.639402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.675977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 977/1939] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:20.678584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.700239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3769de9-eae2-464d-9673-a9f76649ee31 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:20.789866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd0a69-2e11-45e5-945f-e9afc11dac93 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:20.795713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.796619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.796841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.797269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.799008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:20.799246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.800897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.833602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 963/1940] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:20.849492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.877983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709524f7-04cd-40d8-a7cb-d36f5b0c787b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:20.900469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.901141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.901492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.901857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.902602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.905038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.940242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:20.942345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:20.942863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:20.943397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:20.944503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:20.948316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:20.964080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 964/1941] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:20.965640 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:20.988325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 978/1942] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/users => generated 296 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:20.989283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.032725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97a29-8075-4f63-9ffb-4f90200407c7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.053385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:21.081872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696ddd-dc4f-40f6-b577-cd2b861a0a95 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.104094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.105178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.105660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.106149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.113641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:21.160284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.161031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.161386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.161745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.162911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.165228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.217165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 979/1943] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:20 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:21.218903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.234462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc4046-d20f-4480-ad87-93d239c2139d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.242816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.243757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.244240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.244702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.246007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.247331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.249056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.250347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.261469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:21.270068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 980/1944] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:21.271684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.274479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 965/1945] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:21.290374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fd5fc-c045-4f6a-9f1d-d6c4439a281b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.294043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.306568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.307898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.308455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.309048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.310463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.313442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.325977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7e6-1b84-4441-a6ff-6d23c5bee9bc service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.332740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 981/1946] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:21.350026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.352733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.353461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.353920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.354239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] PATH_INFO: `/v3/projects/bded695d5a304d40982bc3b66891c832/users/2a0c46e7e3a14c14923727ab098dcac0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.370215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed34-6cf1-4f07-82e3-bca47decb3e6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.375485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bded695d5a304d40982bc3b66891c832, user_id=2a0c46e7e3a14c14923727ab098dcac0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.378906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.380557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.381642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.382125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.382600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.384249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.387339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.397864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 966/1947] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:21 2022] PUT /identity/v3/projects/bded695d5a304d40982bc3b66891c832/users/2a0c46e7e3a14c14923727ab098dcac0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:21.404159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.419317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 982/1948] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:21.425357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.448578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb857-ee53-4a20-87eb-3bf2e7dd58e3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.542879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaeef8-4e01-425d-8f66-e45eef020627 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.551811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.552735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.553226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.553897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.554590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.556247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.586552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 967/1949] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:21.592958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.594379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.595775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.596577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.597713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] PATH_INFO: `/v3/projects/bdda487459f9498982d40f0881da0540/users/f7a97895b4a84483b9dcb4bc84592c41/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.608947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb765f9e-f75a-4998-9f6a-1406482d5e2d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:21.617674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:21.618476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.618759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.619045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] PATH_INFO: `/v3/projects/bded695d5a304d40982bc3b66891c832/users/2a0c46e7e3a14c14923727ab098dcac0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.624000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bded695d5a304d40982bc3b66891c832, user_id=2a0c46e7e3a14c14923727ab098dcac0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.627965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120c8d-eb81-4a08-8ea9-ad5c26e4ad3c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.636353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdda487459f9498982d40f0881da0540, user_id=f7a97895b4a84483b9dcb4bc84592c41, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:21.650649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8264e-d35a-47b8-9629-9ff515bebf24 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:21.660219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 968/1950] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:21 2022] PUT /identity/v3/projects/bded695d5a304d40982bc3b66891c832/users/2a0c46e7e3a14c14923727ab098dcac0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:21.670042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.670564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.671642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.685460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 983/1951] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:21 2022] PUT /identity/v3/projects/bdda487459f9498982d40f0881da0540/users/f7a97895b4a84483b9dcb4bc84592c41/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:21.690920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:21.691395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:21.692387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:21.723882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:21.725660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:21.729910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] MFA Rules not processed for user `2a0c46e7e3a14c14923727ab098dcac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:21.731937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:21.844704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:21.879150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 984/1952] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:21.891056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:21.895513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:21.912814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 969/1953] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:21.943955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:22.063967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:22.066539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f6573-f0b3-46a3-ae22-3bad04cf1e46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:22.075876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:22.078746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:22.079627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:22.080239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:22.088839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3602ea-00de-41cf-9cbe-bd9bc4bfec9c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:22.096346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:22.097828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:22.098733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:22.099412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:22.104857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:22.108477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:22.118328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:22.120752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:22.150251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 970/1954] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:22.156081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 985/1955] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:22.891575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:22.907985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24578b-7cbc-439f-aa03-0d8bc630ca97 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:22.915419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0c46e7e3a14c14923727ab098dcac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bded695d5a304d40982bc3b66891c832', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:22.916496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:22.916978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:22.917474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:22.918801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:22.919364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:22.922486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:22.944889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 971/1956] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:22.951857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:22.971763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a2fef-3995-4d2d-9a6d-67b78eceeb89 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:22.981917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0c46e7e3a14c14923727ab098dcac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bded695d5a304d40982bc3b66891c832', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:22.984345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:22.984345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:22.984345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:22.986147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:22.990870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:23.037453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 986/1957] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:22 2022] POST /identity/v3/projects => generated 354 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:23.042437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:23.056350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9863-718b-4be7-acf2-f162963b2ccc tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:23.198621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0c46e7e3a14c14923727ab098dcac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bded695d5a304d40982bc3b66891c832', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:23.200080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:23.200627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:23.201163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:23.204583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:23.205413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:23.214006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:23.240664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 972/1958] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:23.310675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:23.338367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37714-c462-484e-8f93-24b82300ac17 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:23.346879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0c46e7e3a14c14923727ab098dcac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bded695d5a304d40982bc3b66891c832', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:23.347691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:23.348130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:23.348626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] PATH_INFO: `/v3/projects/774d7f0dfc5c4a74b3af61bf9cd02b6b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:23.351036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorizing `identity:delete_project(project_id=774d7f0dfc5c4a74b3af61bf9cd02b6b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:23.354869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:23.501993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774d7f0dfc5c4a74b3af61bf9cd02b6b', 'request_id': 'req-e316ada6-5712-4e42-be13-88781d9af17f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:23.531207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774d7f0dfc5c4a74b3af61bf9cd02b6b', 'request_id': 'req-e316ada6-5712-4e42-be13-88781d9af17f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:23.543600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774d7f0dfc5c4a74b3af61bf9cd02b6b', 'request_id': 'req-e316ada6-5712-4e42-be13-88781d9af17f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:23.565907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] The token cache is being invalidate because project 774d7f0dfc5c4a74b3af61bf9cd02b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:23.566510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:23.567439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e316ada6-5712-4e42-be13-88781d9af17f tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:23.570364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 987/1959] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:23 2022] DELETE /identity/v3/projects/774d7f0dfc5c4a74b3af61bf9cd02b6b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:24.007511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:24.011180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:24.031837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:24.032423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:24.032836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:24.080046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] MFA Rules not processed for user `f7a97895b4a84483b9dcb4bc84592c41`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:24.083575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:24.199791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e72b-83d6-4db1-bef0-5336d802f205 tempest-VolumeQuotaClassesTest-988785581 tempest-VolumeQuotaClassesTest-988785581-project-admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:24.219596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:24.285066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 988/1960] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:24.286525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:24.288186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:24.413608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7245-32af-4182-aba8-e9d3c156efc3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:24.426575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:24.428475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:24.428931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:24.429397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] PATH_INFO: `/v3/users/d0ab9fbdfbd54878ae227c661b1d5f89` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:24.433910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:24.436723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:24.438467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ab9fbdfbd54878ae227c661b1d5f89)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:24.439616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:24.440343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:24.444247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:24.447533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:24.451065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:24.528220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:24.546144 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 989/1961] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:24 2022] POST /identity/v3/projects => generated 373 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:24.552821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:24.659905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ab9fbdfbd54878ae227c661b1d5f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:24.663682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ab9fbdfbd54878ae227c661b1d5f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:24.719675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ab9fbdfbd54878ae227c661b1d5f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:24.744528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ab9fbdfbd54878ae227c661b1d5f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:24.747836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ab9fbdfbd54878ae227c661b1d5f89'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:24.846053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 973/1962] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:24 2022] DELETE /identity/v3/users/d0ab9fbdfbd54878ae227c661b1d5f89 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:24.847165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:24.850283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:24.867285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205414b-4f84-40f6-a812-2a8f86ad2782 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.027055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86564b3f-bb93-4beb-bd05-b5a6aab8501a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.027998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.028829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.029108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.029429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.030233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.032404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c498a-708a-4adc-a603-cee62ff69231 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.052299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.053393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.058626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.059892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] PATH_INFO: `/v3/users/7a0763c596354ca083a3ced4aa274f64` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.064068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0763c596354ca083a3ced4aa274f64)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.070305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.076613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 990/1963] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:24 2022] POST /identity/v3/users => generated 293 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:25.079148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.079440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.081222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.124965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:25.128910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.142073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 991/1964] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:25.143810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.146271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.161963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:25.171554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16af7d7-edfb-4939-afa8-2de988d1bb66 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.182195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.183439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.184018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.184598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.186591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.189435 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.217712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 992/1965] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:25.217712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.218065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.232316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0763c596354ca083a3ced4aa274f64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:25.233141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0763c596354ca083a3ced4aa274f64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:25.243624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0763c596354ca083a3ced4aa274f64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:25.260331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0763c596354ca083a3ced4aa274f64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:25.261809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0763c596354ca083a3ced4aa274f64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:25.296082 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ecea-2cf9-44e0-ba20-4c49d068d81c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.300676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.301065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 974/1966] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:24 2022] DELETE /identity/v3/users/7a0763c596354ca083a3ced4aa274f64 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:25.301572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.301914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.302100 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.302232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.304870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.305077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.326946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.328786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92699d1a-d611-45c9-a373-c16b43632b0e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.335440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:25.341932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 993/1967] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:25.344064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.345232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.345232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.374024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:25.375524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.483624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 994/1968] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:25.494244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c2a9-381d-4d1f-a1b4-f913c487dd8f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.503178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.504411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.504877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.506776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.507661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:25.508655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] PATH_INFO: `/v3/projects/940c487056854d8386576adc53e02956/users/7cb9438be25f46709fefce8523e6caae/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.554980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940c487056854d8386576adc53e02956, user_id=7cb9438be25f46709fefce8523e6caae, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.561619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.607925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 975/1969] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:25 2022] PUT /identity/v3/projects/940c487056854d8386576adc53e02956/users/7cb9438be25f46709fefce8523e6caae/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:25.614047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.642244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb771f5-d8ce-474c-bfdc-8a5bebcf0477 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.749799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.750993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.751265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.751537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.756914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:25.816880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b900e8f-e029-4db8-a304-0c32d8ff6c19 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.822960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.823962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.824463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.824917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.825959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.828239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.851242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 976/1970] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:25.859300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:25.890423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c357-ebe9-46ae-9f60-b2ff9e5d9680 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:25.894909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.898279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.926014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:25.927180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:25.927708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:25.928164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] PATH_INFO: `/v3/projects/940c487056854d8386576adc53e02956/users/7cb9438be25f46709fefce8523e6caae/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:25.929647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 995/1971] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:25.931447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] RBAC: Authorizing `identity:create_grant(project_id=940c487056854d8386576adc53e02956, user_id=7cb9438be25f46709fefce8523e6caae, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:25.934963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:25.956699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 977/1972] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:25 2022] PUT /identity/v3/projects/940c487056854d8386576adc53e02956/users/7cb9438be25f46709fefce8523e6caae/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:26.113873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:26.119770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:26.207594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fd84b-b4c1-4976-a592-4447da9c3ce0 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:26.378049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:26.379165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:26.379844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:26.380342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] PATH_INFO: `/v3/projects/cdd7110c8e124d668a035bb88f7baf95` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:26.383614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd7110c8e124d668a035bb88f7baf95)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:26.388691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:26.511440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd7110c8e124d668a035bb88f7baf95', 'request_id': 'req-1cd2228f-94a3-46a7-9122-fac45fb6e39f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.535308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd7110c8e124d668a035bb88f7baf95', 'request_id': 'req-1cd2228f-94a3-46a7-9122-fac45fb6e39f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.559590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd7110c8e124d668a035bb88f7baf95', 'request_id': 'req-1cd2228f-94a3-46a7-9122-fac45fb6e39f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.581659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] The token cache is being invalidate because project cdd7110c8e124d668a035bb88f7baf95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:26.581948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.582731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.584355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 996/1973] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:26 2022] DELETE /identity/v3/projects/cdd7110c8e124d668a035bb88f7baf95 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:26.591151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:26.594199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:26.604012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:26.612280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:26.668329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdd83-6fd6-4610-a4e9-4a210255607b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:26.751513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:26.832412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2228f-94a3-46a7-9122-fac45fb6e39f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:26.845335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:26.846042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:26.846320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:26.846602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] PATH_INFO: `/v3/users/2a0c46e7e3a14c14923727ab098dcac0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:26.847548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:26.848212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:26.848478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:26.848748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] PATH_INFO: `/v3/projects/56228c9407654b649d1133df5553adf3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:26.851015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0c46e7e3a14c14923727ab098dcac0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:26.853003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:26.855867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=56228c9407654b649d1133df5553adf3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:26.857868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:26.937385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:26.984101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0c46e7e3a14c14923727ab098dcac0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.984914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0c46e7e3a14c14923727ab098dcac0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.991653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56228c9407654b649d1133df5553adf3', 'request_id': 'req-01a3d2f0-2510-4439-9930-b882b416d0ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:26.995877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0c46e7e3a14c14923727ab098dcac0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.025609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56228c9407654b649d1133df5553adf3', 'request_id': 'req-01a3d2f0-2510-4439-9930-b882b416d0ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.031557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0c46e7e3a14c14923727ab098dcac0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.033391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ed2ae41-d849-4ad4-bd79-abb6f09884eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0c46e7e3a14c14923727ab098dcac0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.052134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56228c9407654b649d1133df5553adf3', 'request_id': 'req-01a3d2f0-2510-4439-9930-b882b416d0ba'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.080280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] The token cache is being invalidate because project 56228c9407654b649d1133df5553adf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:27.081133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.082622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.085808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 997/1974] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:26 2022] DELETE /identity/v3/projects/56228c9407654b649d1133df5553adf3 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:27.093705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:27.096769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:27.116911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 978/1975] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:26 2022] DELETE /identity/v3/users/2a0c46e7e3a14c14923727ab098dcac0 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:27.187931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3d2f0-2510-4439-9930-b882b416d0ba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:27.322262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:27.323508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:27.324019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:27.324637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] PATH_INFO: `/v3/users/3a43e158d19b4249b23585693cd983de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:27.327093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a43e158d19b4249b23585693cd983de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:27.331597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:27.355247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:27.451420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a43e158d19b4249b23585693cd983de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.467556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a43e158d19b4249b23585693cd983de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.468586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a43e158d19b4249b23585693cd983de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.470051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a43e158d19b4249b23585693cd983de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.493066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a43e158d19b4249b23585693cd983de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:27.525815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 998/1976] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:27 2022] DELETE /identity/v3/users/3a43e158d19b4249b23585693cd983de => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:27.588838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:27.589303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:27.590430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:27.648105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] MFA Rules not processed for user `7cb9438be25f46709fefce8523e6caae`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:27.651493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:27.808219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 979/1977] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:27.823464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:27.824738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:27.826227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:27.828933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:27.970024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a9da-3dd6-4a99-8b1a-6f9a89dc32b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:27.976346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:27.977402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:27.978958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:27.979582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:27.983315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.069089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a8dfc-ede1-4818-a318-a4128c45a867 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.078456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.079508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.080004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.080387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] PATH_INFO: `/v3/projects/bded695d5a304d40982bc3b66891c832` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.081847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bded695d5a304d40982bc3b66891c832)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.085868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.094786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:28.159019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.162818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.187733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 980/1978] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:28.190137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.193701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.194649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bded695d5a304d40982bc3b66891c832', 'request_id': 'req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.212075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bded695d5a304d40982bc3b66891c832', 'request_id': 'req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.224689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bded695d5a304d40982bc3b66891c832', 'request_id': 'req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.230140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d213-a6d3-4519-8979-ffe0f8de0990 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.246861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] The token cache is being invalidate because project bded695d5a304d40982bc3b66891c832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:28.247364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.248451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b223a693-cb70-49d5-9a70-2fa6d7c5e2e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.266478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 999/1979] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:27 2022] DELETE /identity/v3/projects/bded695d5a304d40982bc3b66891c832 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:28.269006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c61e7-6fe0-4d20-8c81-ef6680a77306 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.269447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c61e7-6fe0-4d20-8c81-ef6680a77306 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.269903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c61e7-6fe0-4d20-8c81-ef6680a77306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.301234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cb0c61e7-6fe0-4d20-8c81-ef6680a77306 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:28.303444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cb0c61e7-6fe0-4d20-8c81-ef6680a77306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.330918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:28.348463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.348779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.349103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.349353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] PATH_INFO: `/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.352525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a9af4901c94b068ae46dd4f9fe2d5b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.355219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.462572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1000/1980] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:28.467166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.467596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.468015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.500356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:28.502417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.518972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a9af4901c94b068ae46dd4f9fe2d5b', 'request_id': 'req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.536930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a9af4901c94b068ae46dd4f9fe2d5b', 'request_id': 'req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.549332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a9af4901c94b068ae46dd4f9fe2d5b', 'request_id': 'req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.566713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] The token cache is being invalidate because project 55a9af4901c94b068ae46dd4f9fe2d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:28.567287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.568381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fd621d2e-8c31-4fdc-91c9-5e5a6d88305c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:28.571958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 981/1981] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:28 2022] DELETE /identity/v3/projects/55a9af4901c94b068ae46dd4f9fe2d5b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:28.577712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a1bf8-6f1c-48b3-b224-9ce76000b4f9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.578875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a1bf8-6f1c-48b3-b224-9ce76000b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.578875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a1bf8-6f1c-48b3-b224-9ce76000b4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.607393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1001/1982] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:28.615123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.615641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0e5a1bf8-6f1c-48b3-b224-9ce76000b4f9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:28.617187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0e5a1bf8-6f1c-48b3-b224-9ce76000b4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.632808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9a40-eefc-4aaf-a630-a63b3afe2aa6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.634669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 982/1983] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:28.645341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.646829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.647288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.647828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.649208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:28.649594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.650311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.650505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.650505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.653450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.698211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1002/1984] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:44:28.698492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:28.698492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:28.700535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.720866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 983/1985] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:28.722516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.734456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fd972-38a0-4d63-811a-85b020ddc003 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.755452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.756504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.756904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.757372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.758492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.762776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.764836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1276-5b38-46a0-97e6-895356d70852 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.779332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.780908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.781397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.782149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.783628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:28.784189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.788121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:28.820163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 984/1986] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:28.823348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.828867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1003/1987] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:28.834768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:28.844928 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c0000-3bc9-4b96-a198-4071e4fd7e28 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:28.932330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:28.993276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:28.993770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:28.993932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:28.994331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:28.994968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:28.996925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.054608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49612-34e7-4608-8879-ede7a6b03b40 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.065946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.067884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.068187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.068499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.069415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.081843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.103471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 985/1988] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:29.108900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.160738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1004/1989] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:28 2022] POST /identity/v3/users => generated 289 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:29.167000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.183119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84500e7-7bb2-4fe9-9184-bef55097d211 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.203230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:29.236873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66576aaf-f244-47de-9b2f-6459b45112cb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.251091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.258353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.262492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.263123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.264326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.268711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.327781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.329967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.330491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.331440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.332487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 986/1990] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:29 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:29.332814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.334466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.336859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.357140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b0de3-6a96-41ae-9665-2913d35ab2be admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.367871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1005/1991] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:29.376807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.380617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.383058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.383710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.384286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.385757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.389670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.398110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad43f0-a950-497c-8587-014d6093d514 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.413006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 987/1992] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:29.415508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.416566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.418229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.418781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.419998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.422409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.426313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.430075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce715ad-73c2-41e2-bc50-b30253877319 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.436389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.437218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.437685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.438762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.439549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.441359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.455893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1006/1993] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:29.461719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.469803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 988/1994] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:29.472252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.480852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b671deb-9545-4add-a9e9-f9f763ef526b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.489209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.491259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.491259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.491498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] PATH_INFO: `/v3/projects/3c11e507868f42ea9c165a1b05ac445b/users/d7cee08fc5b848f69fbe022f78c1c9d8/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.495221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c894f-3702-4ba9-8970-05628f026b13 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:29.505951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:29.507277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.507756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.508221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] PATH_INFO: `/v3/projects/ae6999cd3cf1454494272fb4fc103f83/users/38f086dca51c49a5b6e0db740b760f29/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.533302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c11e507868f42ea9c165a1b05ac445b, user_id=d7cee08fc5b848f69fbe022f78c1c9d8, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.541385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081ca49-88b2-47d1-a1ec-a4c12e4d87d3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.563463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6999cd3cf1454494272fb4fc103f83, user_id=38f086dca51c49a5b6e0db740b760f29, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:29.581594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1007/1995] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:29 2022] PUT /identity/v3/projects/3c11e507868f42ea9c165a1b05ac445b/users/d7cee08fc5b848f69fbe022f78c1c9d8/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:29.582852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05655ad-1f36-4119-b5b4-94b9ec77225c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:29.589148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.591508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.592009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.615126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 989/1996] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:29 2022] PUT /identity/v3/projects/ae6999cd3cf1454494272fb4fc103f83/users/38f086dca51c49a5b6e0db740b760f29/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:29.634800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:29.635413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:29.636023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:29.677873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] MFA Rules not processed for user `d7cee08fc5b848f69fbe022f78c1c9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:29.679889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:29.688727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] MFA Rules not processed for user `38f086dca51c49a5b6e0db740b760f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:29.690151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:29.836777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:29.848872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 990/1997] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:29.855038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:29.903134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1008/1998] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:29.911146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.047516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3599e0-755e-4503-a89c-a4927952b46a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.062203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.063656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.064226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.064539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059f7cb-ad78-4b49-85f5-959e4b2caace None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.064908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.067210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.070840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.071188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.071488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.073241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.073548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.074828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.076999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.095155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:30.131451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1009/1999] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:29 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:30.138037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.144849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 991/2000] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:29 2022] POST /identity/v3/projects => generated 379 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:30.147708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.288503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06946f-185d-41fc-981f-917b87a713db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.296129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.296823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.297107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.297375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.298091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.300071 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.305833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f7eb-a7cb-414e-8b27-7dbc99c3ec47 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.311641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.312803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.313304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.313840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.315337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.318440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.325092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 992/2001] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:30.325092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.325092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:30.352660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1010/2002] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:30.352660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.367636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5569f-24a3-471e-bf62-9285cac81589 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.376600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.378409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.378409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.378681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.382986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.382986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.398098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1011/2003] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:30.399370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.424290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74d273-d9d6-4436-9f5c-195691d11006 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.426728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f848e-06f7-4ede-a535-9d4c735bb2d9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.431435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.431696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.432272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.432475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.432720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.432941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.433185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.433347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.434462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.435483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:30.437203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.453015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1012/2004] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:30.457353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.469334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde1e27-1adc-42a6-bb5e-4e679901ba27 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.474556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.475205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.475512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.475789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.476505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.477951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:30.478148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.490661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1013/2005] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:30.491612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.505352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b1145-fb47-40b0-85ed-8b2c48d1c1f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.514827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.515913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.516390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.516862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.518423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.522785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.538367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.541197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.543693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1014/2006] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:30.543923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.560448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f8dd1-2430-4b25-a7cb-a45b27523fa7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.566864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 993/2007] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:30.569044 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.569310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.570215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.570665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.571216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] PATH_INFO: `/v3/projects/cc75275cd69542428c175a56f3be76a2/users/b6725a8fc28d4ef5b8b812cc5d4271a7/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.593004 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca60ef4-0d76-401a-9f1b-ee33ac283c52 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:30.600747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:30.601098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc75275cd69542428c175a56f3be76a2, user_id=b6725a8fc28d4ef5b8b812cc5d4271a7, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.601891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.602559 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.603020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] PATH_INFO: `/v3/projects/707ec079cb184ae1a1cb248dbc64a4f0/users/2aaa35f3b1744568b9ec9ba6fcd82b04/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.606716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17a2ea-f18b-4ce4-a39d-8003f66a47ae admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.631668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1015/2008] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:30 2022] PUT /identity/v3/projects/cc75275cd69542428c175a56f3be76a2/users/b6725a8fc28d4ef5b8b812cc5d4271a7/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:30.635380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=707ec079cb184ae1a1cb248dbc64a4f0, user_id=2aaa35f3b1744568b9ec9ba6fcd82b04, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:30.641908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e5dc2-a356-485c-b7eb-f4b3e9f9a48b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:30.650266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.650992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.651421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.672540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 994/2009] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:30 2022] PUT /identity/v3/projects/707ec079cb184ae1a1cb248dbc64a4f0/users/2aaa35f3b1744568b9ec9ba6fcd82b04/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:30.684468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:30.691538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:30.692447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:30.735769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] MFA Rules not processed for user `b6725a8fc28d4ef5b8b812cc5d4271a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:30.738888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:30.755394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] MFA Rules not processed for user `2aaa35f3b1744568b9ec9ba6fcd82b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:30.757716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:30.882510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 995/2010] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:30.887814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:30.892977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:30.909767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1016/2011] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:30.911084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.044737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f615c-e68c-4bfa-9981-84a8c03aed3b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.059303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.060706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.061203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.061831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.063213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.067366 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.138759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77efc56-4bb5-42f2-9f0c-8be19d6c02bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.151077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.152726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.152726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.153228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.154388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.157848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 996/2012] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/projects => generated 377 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:31.158617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.173654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.210679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:31.214464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1017/2013] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:30 2022] POST /identity/v3/projects => generated 379 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:31.222469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.336582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dc5e7-a9da-4a77-8ec8-349f1acff156 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.342465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf585a3-a0e2-4f93-809e-644655cabba2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.346352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.347749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.348242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.348695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.349673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.350025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.350326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.350612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.350901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.351538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.353469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.353692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.384721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1018/2014] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:31 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:31.387354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 997/2015] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:31 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:31.390907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.392329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.404956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53fc8c-612b-46f6-86ba-6011764cce02 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.408316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0549f-fa82-4896-851c-078c48e69447 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.409975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.410614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.410898 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.411171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.411916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.413934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.417689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.418639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.419146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.419658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.420916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.423024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.430957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 998/2016] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:31.434655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1019/2017] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:31.438236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.438979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.450480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd0ca3-6b2a-4544-b0aa-72910254122c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.453395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e740-7fe8-4ee3-bbe7-8306e8dd3cde admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.456484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99553b67-518f-4601-a286-70813e953395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.457406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99553b67-518f-4601-a286-70813e953395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.457892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99553b67-518f-4601-a286-70813e953395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.458368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99553b67-518f-4601-a286-70813e953395 admin admin] PATH_INFO: `/v3/projects/5b1b9217b737400e808bd3e1737d25b4/users/c59f70f7ca76405592f163049225bbae/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.458854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.459577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.460084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.460247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] PATH_INFO: `/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2/users/75d9b7fb705a4ecd9038177a3f9364bf/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.479179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99553b67-518f-4601-a286-70813e953395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1b9217b737400e808bd3e1737d25b4, user_id=c59f70f7ca76405592f163049225bbae, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.481918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99553b67-518f-4601-a286-70813e953395 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.492701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b3dfab903f147dba042ebd4dbc0c8b2, user_id=75d9b7fb705a4ecd9038177a3f9364bf, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.497457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.503876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1020/2018] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:31 2022] PUT /identity/v3/projects/5b1b9217b737400e808bd3e1737d25b4/users/c59f70f7ca76405592f163049225bbae/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:31.510051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99553b67-518f-4601-a286-70813e953395 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.522113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 999/2019] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:31 2022] PUT /identity/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2/users/75d9b7fb705a4ecd9038177a3f9364bf/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:31.527130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.610502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:31.644125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797216-cb74-4842-908a-724f8ee2e72a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.644637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99553b67-518f-4601-a286-70813e953395 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.652079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.653109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.653654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.654070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.655244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.655495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.656861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.657669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.658293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.658615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.660222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.663685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.677673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1000/2020] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:31.684678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1021/2021] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:31.689302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.693770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.700356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdaed2-18f2-45c9-9a1f-53fe0e8e2166 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.706092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.706953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.708073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.708073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] PATH_INFO: `/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2/users/75d9b7fb705a4ecd9038177a3f9364bf/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.711003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad94c-e310-4653-8ff5-1d3c203fba9f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:31.711627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b3dfab903f147dba042ebd4dbc0c8b2, user_id=75d9b7fb705a4ecd9038177a3f9364bf, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.719439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2373e-c25c-43bb-98ba-e4e1277ab95b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.723796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:31.724768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.725455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.726368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] PATH_INFO: `/v3/projects/5b1b9217b737400e808bd3e1737d25b4/users/c59f70f7ca76405592f163049225bbae/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.729657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1b9217b737400e808bd3e1737d25b4, user_id=c59f70f7ca76405592f163049225bbae, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:31.733169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a8100-1dc9-4352-afee-eee10b6bf259 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:31.737817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1001/2022] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:31 2022] PUT /identity/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2/users/75d9b7fb705a4ecd9038177a3f9364bf/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:31.751032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.751314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.751750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.757665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1022/2023] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:31 2022] PUT /identity/v3/projects/5b1b9217b737400e808bd3e1737d25b4/users/c59f70f7ca76405592f163049225bbae/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:31.763315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:31.763315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:31.763980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:31.797581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] MFA Rules not processed for user `75d9b7fb705a4ecd9038177a3f9364bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:31.799051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:31.831491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] MFA Rules not processed for user `c59f70f7ca76405592f163049225bbae`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:31.833497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:31.868063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:31.915580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1002/2024] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:31.929587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.966063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1023/2025] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:31.986188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:31.988145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:31.996953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac15d8-cb37-4da9-a497-c241048bd4a9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:32.002605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:32.003642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:32.004116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:32.004569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:32.007387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64b0b9-b30b-44af-8bd1-d880fa1d662e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:32.019584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:32.020352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:32.020814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:32.021247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:32.032561 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:32.034066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:32.045403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1024/2026] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:32.097335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:32.101987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:32.104748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:32.111761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad0c3-41a0-4c8a-bd89-ff1f88a818bf service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:32.118118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:32.119478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:32.120051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:32.124050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:32.124189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1003/2027] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:32.205495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:32.208798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:32.226161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1025/2028] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:34.154955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:34.168768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575e1e-48db-400d-a770-0c42c649b0f7 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:34.172523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:34.173151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:34.173441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:34.174072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:34.272689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:34.274326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:34.293728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1004/2029] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:35.354375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:35.364973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4d112-3e51-4fa9-8df2-dd12bfbb3199 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:35.372292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:35.373599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:35.374053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:35.374616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:35.394319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:35.397045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d5a8c-478e-41e6-9c32-1597e8d77b47 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:35.412737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1026/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:36.399435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:36.484855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:36.536759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7662e-a480-40d2-845e-7692eab4b225 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:36.546124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:36.548118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:36.549261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:36.549921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] PATH_INFO: `/v3/users/38f086dca51c49a5b6e0db740b760f29` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:36.553446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f086dca51c49a5b6e0db740b760f29)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:36.557415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:36.648654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f086dca51c49a5b6e0db740b760f29'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:36.651419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f086dca51c49a5b6e0db740b760f29'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:36.664839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f086dca51c49a5b6e0db740b760f29'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:36.676084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f086dca51c49a5b6e0db740b760f29'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:36.676880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f086dca51c49a5b6e0db740b760f29'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:36.702897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1005/2031] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:36 2022] DELETE /identity/v3/users/38f086dca51c49a5b6e0db740b760f29 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:36.707864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:36.708402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:36.708402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:36.735872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:36.738038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:36.759225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:36.839142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1027/2032] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:36.853220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:36.855098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:36.957775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ba268-580d-4551-acb7-45372aa5d832 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:36.963317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:36.964219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:36.964997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:36.965731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:36.981770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:36.984543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:37.006836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1006/2033] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:37.948242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:37.954384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:37.977792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea559-2bfb-422a-8b85-4780d0c75792 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:37.985709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:37.988693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:37.989259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:37.990540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:37.990989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:37.991745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] PATH_INFO: `/v3/users/c8d88d538a42449292fb5f01da24dd64` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:37.994383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d88d538a42449292fb5f01da24dd64)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:37.997791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:38.039185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d634-00f5-4d4f-af81-0e8c8286671b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:38.044740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:38.045976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:38.046647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:38.047804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] PATH_INFO: `/v3/projects/ae6999cd3cf1454494272fb4fc103f83` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:38.049255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6999cd3cf1454494272fb4fc103f83)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:38.051457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:38.134937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d88d538a42449292fb5f01da24dd64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.142230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:38.145322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6999cd3cf1454494272fb4fc103f83', 'request_id': 'req-2f0ae5d2-003c-4810-a227-0cae840a9c94'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.155519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d88d538a42449292fb5f01da24dd64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.156877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d88d538a42449292fb5f01da24dd64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.158102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d88d538a42449292fb5f01da24dd64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.168036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6999cd3cf1454494272fb4fc103f83', 'request_id': 'req-2f0ae5d2-003c-4810-a227-0cae840a9c94'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.179228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6999cd3cf1454494272fb4fc103f83', 'request_id': 'req-2f0ae5d2-003c-4810-a227-0cae840a9c94'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.189543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3fee447-5600-4f66-8059-17eaed25a5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d88d538a42449292fb5f01da24dd64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.198120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] The token cache is being invalidate because project ae6999cd3cf1454494272fb4fc103f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:38.198313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.199828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.209116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1007/2034] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:37 2022] DELETE /identity/v3/projects/ae6999cd3cf1454494272fb4fc103f83 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:38.212105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:38.214643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1028/2035] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:37 2022] DELETE /identity/v3/users/c8d88d538a42449292fb5f01da24dd64 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:38.215827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:38.224048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379100f-8813-4549-bc25-4001f359c9df None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:38.224340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379100f-8813-4549-bc25-4001f359c9df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:38.224618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379100f-8813-4549-bc25-4001f359c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:38.251854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5379100f-8813-4549-bc25-4001f359c9df None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:38.253202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5379100f-8813-4549-bc25-4001f359c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:38.332728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ae5d2-003c-4810-a227-0cae840a9c94 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:38.376461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1029/2036] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:38.384488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379100f-8813-4549-bc25-4001f359c9df None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:38.386822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5379100f-8813-4549-bc25-4001f359c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:38.428372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:38.429216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:38.429641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:38.429980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] PATH_INFO: `/v3/users/b6725a8fc28d4ef5b8b812cc5d4271a7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:38.432634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6725a8fc28d4ef5b8b812cc5d4271a7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:38.435616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:38.437536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:38.486700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379100f-8813-4549-bc25-4001f359c9df None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:38.496536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:38.497790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:38.498208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:38.498577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:38.523594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:38.525619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:38.535594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6725a8fc28d4ef5b8b812cc5d4271a7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.536264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6725a8fc28d4ef5b8b812cc5d4271a7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.537007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1030/2037] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:38.549726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6725a8fc28d4ef5b8b812cc5d4271a7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.567585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6725a8fc28d4ef5b8b812cc5d4271a7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.568747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6725a8fc28d4ef5b8b812cc5d4271a7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:38.603252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1008/2038] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:38 2022] DELETE /identity/v3/users/b6725a8fc28d4ef5b8b812cc5d4271a7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:39.357523 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:39.386873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764553d-b9e0-4fbe-8b2b-eedf64dd9dc2 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:39.490755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:39.494345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:39.494345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:39.494345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] PATH_INFO: `/v3/projects/cc75275cd69542428c175a56f3be76a2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:39.494345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc75275cd69542428c175a56f3be76a2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:39.496711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:39.575287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:39.578922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:39.601744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc75275cd69542428c175a56f3be76a2', 'request_id': 'req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.618471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:39.628344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc75275cd69542428c175a56f3be76a2', 'request_id': 'req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.644444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc75275cd69542428c175a56f3be76a2', 'request_id': 'req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.648494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d97610-99c8-4cb3-9f0b-dc06bdcdfae2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:39.670004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] The token cache is being invalidate because project cc75275cd69542428c175a56f3be76a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:39.670847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.672078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.675393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1031/2039] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:39 2022] DELETE /identity/v3/projects/cc75275cd69542428c175a56f3be76a2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:39.679674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:39.682573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:39.824752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:39.826233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:39.826896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:39.827608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] PATH_INFO: `/v3/projects/7908dd73c39a4d30ac4b2bd9c69002f8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:39.830923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=7908dd73c39a4d30ac4b2bd9c69002f8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:39.834393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:39.849476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee0859-3a7d-43a0-8eeb-94e6412be9eb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:39.869498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:39.870846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:39.871421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:39.872040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] PATH_INFO: `/v3/users/c59f70f7ca76405592f163049225bbae` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:39.874320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59f70f7ca76405592f163049225bbae)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:39.877870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:39.892120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:39.954747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7908dd73c39a4d30ac4b2bd9c69002f8', 'request_id': 'req-10e95bee-4e76-489c-aa50-2804b7d741ed'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.970111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7908dd73c39a4d30ac4b2bd9c69002f8', 'request_id': 'req-10e95bee-4e76-489c-aa50-2804b7d741ed'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.976132 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59f70f7ca76405592f163049225bbae'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.984666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7908dd73c39a4d30ac4b2bd9c69002f8', 'request_id': 'req-10e95bee-4e76-489c-aa50-2804b7d741ed'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.994303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59f70f7ca76405592f163049225bbae'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.995310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59f70f7ca76405592f163049225bbae'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:39.996007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59f70f7ca76405592f163049225bbae'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.010902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] The token cache is being invalidate because project 7908dd73c39a4d30ac4b2bd9c69002f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:40.012791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.012791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-10e95bee-4e76-489c-aa50-2804b7d741ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.016725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1009/2040] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:39 2022] DELETE /identity/v3/projects/7908dd73c39a4d30ac4b2bd9c69002f8 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:40.024878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78f9d9-2576-4555-bc3d-3ff7d61d0bc8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.024878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78f9d9-2576-4555-bc3d-3ff7d61d0bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.024878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78f9d9-2576-4555-bc3d-3ff7d61d0bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.062377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59f70f7ca76405592f163049225bbae'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.075666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5b78f9d9-2576-4555-bc3d-3ff7d61d0bc8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:40.077132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b78f9d9-2576-4555-bc3d-3ff7d61d0bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:40.102253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1032/2041] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:39 2022] DELETE /identity/v3/users/c59f70f7ca76405592f163049225bbae => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:40.129350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:40.133272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:40.314555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1010/2042] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:40.333668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.333668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.333668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.365356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b185516-9f9f-46a0-b2a6-5da9f7a9cad8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:40.376370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:40.377213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.378551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.379037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] PATH_INFO: `/v3/users/d7cee08fc5b848f69fbe022f78c1c9d8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.382314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7cee08fc5b848f69fbe022f78c1c9d8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:40.383714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:40.385726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:40.386160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:40.406376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1011/2043] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:40.410897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:40.442125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c0bcd-2d9a-4605-9c34-f2ff3c6374b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:40.455090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:40.456451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.457096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.457699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.459268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:40.460173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:40.463419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:40.490744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1012/2044] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:44:40.506042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:40.539692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d758a-bfea-42c1-b50c-2945ce44caf4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:40.554685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:40.555839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.555839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.556251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.557412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:40.562168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:40.572418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7cee08fc5b848f69fbe022f78c1c9d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.593083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:40.626662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7cee08fc5b848f69fbe022f78c1c9d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.630332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7cee08fc5b848f69fbe022f78c1c9d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.634619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7cee08fc5b848f69fbe022f78c1c9d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.669032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1013/2045] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:40 2022] POST /identity/v3/projects => generated 359 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:40.671086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:40.681055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:40.696895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9afef1f6-fb6c-4cd8-90be-376daa589bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7cee08fc5b848f69fbe022f78c1c9d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:40.756956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1033/2046] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:40 2022] DELETE /identity/v3/users/d7cee08fc5b848f69fbe022f78c1c9d8 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:40.769036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:40.770581 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:40.772187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:40.801223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:40.866901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5ebf-dcdc-408e-99b7-3c45e4acdd0e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:40.868372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:40.870757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.002832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.004328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.004737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.005083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.005882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.008372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0eafa-9ceb-4032-ab27-aa331743e5bb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.017263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1034/2047] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:41.018550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.020578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.040864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8460bbf-169d-4418-bd25-21d8e4957c4a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.047233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1014/2048] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:40 2022] POST /identity/v3/users => generated 280 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:41.048909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.050357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.050552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.051008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] PATH_INFO: `/v3/projects/5b1b9217b737400e808bd3e1737d25b4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.053003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.053127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1b9217b737400e808bd3e1737d25b4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.053298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.053662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.055042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1015/2049] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:44:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:44:41.056294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.058476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.063639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.086205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c2946-a92f-412c-9cc1-ba7ecd29fa94 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.093620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.094703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.095200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.095691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] PATH_INFO: `/v3/users/a2ff5f41f9b44c2c9f5f90f13c266fbd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.097743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ff5f41f9b44c2c9f5f90f13c266fbd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.101325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.140067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1b9217b737400e808bd3e1737d25b4', 'request_id': 'req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.151447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1b9217b737400e808bd3e1737d25b4', 'request_id': 'req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.161471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1b9217b737400e808bd3e1737d25b4', 'request_id': 'req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.172397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ff5f41f9b44c2c9f5f90f13c266fbd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.174365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ff5f41f9b44c2c9f5f90f13c266fbd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.178840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] The token cache is being invalidate because project 5b1b9217b737400e808bd3e1737d25b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:41.179224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.179998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.181867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1035/2050] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:41 2022] DELETE /identity/v3/projects/5b1b9217b737400e808bd3e1737d25b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:41.182996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.184773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.186722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ff5f41f9b44c2c9f5f90f13c266fbd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.195895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ff5f41f9b44c2c9f5f90f13c266fbd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.197188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ff5f41f9b44c2c9f5f90f13c266fbd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.217071 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:41.224462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1016/2051] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:41 2022] DELETE /identity/v3/users/a2ff5f41f9b44c2c9f5f90f13c266fbd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:41.225818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.227682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.284315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4d5c6-223c-4c87-b1b8-7fd2524841f0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.370423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4caa1-9725-4851-b6e1-59f7924c8fb6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.375705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:41.380445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.381436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.381982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.382592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.383961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.388567 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d399c-fc59-4f41-894c-9dff87047f00 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.392541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.393417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.393728 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.393997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.395950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.410975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.416780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dd463-3569-4371-9d1f-99ff4b94b360 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.416995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1017/2052] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:41.416995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.416995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.416995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.425366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1036/2053] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:41.426411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.426693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.426965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.444518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:41.445782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.456450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:41.458521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.471284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1037/2054] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:41.472104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.480422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.494863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388c368-8c22-4bd8-b7a5-070e295dd277 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.500341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.501197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.501610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.502371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.503071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.507362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ca9e4-5fe7-4a14-b0a7-fdc0207d8fc1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.519935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1038/2055] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:41.519935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.519935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.519935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.546094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:41.547976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.559269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1018/2056] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:44:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:41.559908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.572456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1039/2057] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:41.574248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.575656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2e000-def7-46ae-b671-6b57134cb870 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.583523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.584855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.584855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.584855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] PATH_INFO: `/v3/projects/61ea1dce6cf54ebd8d62c202f2aa0520/users/7a8ebdfe484945389cec748cf4eafb18/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.596805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad8932-d05d-417d-a6c0-0c140b7eef06 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.603960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.604606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.605475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.605475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.606415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:41.606415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.608084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.612265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ea1dce6cf54ebd8d62c202f2aa0520, user_id=7a8ebdfe484945389cec748cf4eafb18, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.615702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.636683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1019/2058] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:41 2022] PUT /identity/v3/projects/61ea1dce6cf54ebd8d62c202f2aa0520/users/7a8ebdfe484945389cec748cf4eafb18/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:41.637602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.651764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.668140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1040/2059] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:41.669915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.673349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.707957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f13d-fac8-43a9-ad4b-ddad600b6696 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.788264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0090a-d53a-4f38-81f4-bb8f5be3058a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:41.790324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.791105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.791465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.791795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.796470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:41.797725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.798244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.798804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] PATH_INFO: `/v3/projects/41e72072c5cc453fbc31425b050a07c1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.800927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e72072c5cc453fbc31425b050a07c1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.803088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:41.804689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0d89e-afdd-48ec-b571-5d7c1704432a service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.804870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:41.818699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1041/2060] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:41.819829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:41.820115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:41.820382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:41.825448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:41.868312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] MFA Rules not processed for user `7a8ebdfe484945389cec748cf4eafb18`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:41.870994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:41.902707 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e72072c5cc453fbc31425b050a07c1', 'request_id': 'req-71ef0fcd-3ced-48f2-9856-169200c24fe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.914436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e72072c5cc453fbc31425b050a07c1', 'request_id': 'req-71ef0fcd-3ced-48f2-9856-169200c24fe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.925402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e72072c5cc453fbc31425b050a07c1', 'request_id': 'req-71ef0fcd-3ced-48f2-9856-169200c24fe7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.957541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] The token cache is being invalidate because project 41e72072c5cc453fbc31425b050a07c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:41.958368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.959555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:41.962284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1020/2061] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:41 2022] DELETE /identity/v3/projects/41e72072c5cc453fbc31425b050a07c1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:41.963577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:41.966260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.019371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1042/2062] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:41 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:42.020423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.024385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.060525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a730635-d04a-4f57-af02-f98a151264e6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.095099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0fcd-3ced-48f2-9856-169200c24fe7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.104137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.106178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.106512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:42.106749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.107266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.108355 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.112361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ab108-e008-45db-ad89-3627b8c2d151 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.167549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1021/2063] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:41 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:42.167549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.167549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.167549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.178470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.180451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.181128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.181992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] PATH_INFO: `/v3/projects/3c11e507868f42ea9c165a1b05ac445b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.184404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c11e507868f42ea9c165a1b05ac445b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.188415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.198391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:42.199736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.291660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c11e507868f42ea9c165a1b05ac445b', 'request_id': 'req-f2519e19-3522-4794-a874-348367679f95'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.304218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c11e507868f42ea9c165a1b05ac445b', 'request_id': 'req-f2519e19-3522-4794-a874-348367679f95'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.316756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c11e507868f42ea9c165a1b05ac445b', 'request_id': 'req-f2519e19-3522-4794-a874-348367679f95'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.326813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1022/2064] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:42.327902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.329770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.337830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] The token cache is being invalidate because project 3c11e507868f42ea9c165a1b05ac445b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:42.338395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.340421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.343007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1043/2065] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:42 2022] DELETE /identity/v3/projects/3c11e507868f42ea9c165a1b05ac445b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:42.344259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.346439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.416091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:42.451273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519e19-3522-4794-a874-348367679f95 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.457528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.458373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.458800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.459071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.459755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.461766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450c7a5-77db-4e9b-b137-1cd8807df9c0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.475869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c864a-acf1-4c0c-9803-3df1df015262 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.500734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1044/2066] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:42 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:42.502476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.503083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.503526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.525204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:42.526818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.541597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1045/2067] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:42.541597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.541597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.561080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7253-ad97-4eb5-9d89-410faf2a89ca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.561956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.562750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.563228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.563594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.565689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.568291 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.569458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.569964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.570369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] PATH_INFO: `/v3/users/2aaa35f3b1744568b9ec9ba6fcd82b04` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.572605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aaa35f3b1744568b9ec9ba6fcd82b04)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.576630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.630621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:42.646998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aaa35f3b1744568b9ec9ba6fcd82b04'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.653127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.654656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.661635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aaa35f3b1744568b9ec9ba6fcd82b04'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.662244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aaa35f3b1744568b9ec9ba6fcd82b04'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.663031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aaa35f3b1744568b9ec9ba6fcd82b04'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.668795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1023/2068] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:42 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:42.669899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.693082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9395bb0-3b83-4e2a-a977-304dd9090225 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.695890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aaa35f3b1744568b9ec9ba6fcd82b04'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.699473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.700371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.700739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.701193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.702059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.703912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.715180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1046/2069] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:42 2022] DELETE /identity/v3/users/2aaa35f3b1744568b9ec9ba6fcd82b04 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:42.716255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.718134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.722083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1024/2070] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:42.725902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.728154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.748634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a404e7-862d-4841-b84a-fa5d5d6ed798 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.854725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb336-b72a-441c-a508-52baf4cc6cd7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.861323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.862594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.863581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.864580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.864706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] PATH_INFO: `/v3/users/7a8ebdfe484945389cec748cf4eafb18` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.864787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.865055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.865283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.866244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:42.866573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.866810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8ebdfe484945389cec748cf4eafb18)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.868245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.870775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.890453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1047/2071] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:42.890453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.893532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:42.921081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc19cba-481f-4d04-aa76-51a24f883f3d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.924972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:42.929313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.930451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.930841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.932957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.932957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:42.936174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:42.964530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1048/2072] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:42.964530 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:42.979909 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8ebdfe484945389cec748cf4eafb18'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.981392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8ebdfe484945389cec748cf4eafb18'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.982956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94786be-2408-46e8-a4d8-3d14ff32bb5e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:42.991314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8ebdfe484945389cec748cf4eafb18'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:42.991628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:42.992806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:42.993396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:42.994091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:42.995447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.005739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eb0bf-66be-4d02-961b-74747a563c2f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.009841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8ebdfe484945389cec748cf4eafb18'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.010847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8ebdfe484945389cec748cf4eafb18'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.042651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1025/2073] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:42 2022] DELETE /identity/v3/users/7a8ebdfe484945389cec748cf4eafb18 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:43.048455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.050255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1049/2074] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:42 2022] POST /identity/v3/projects => generated 403 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:43.052598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.054713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.055142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.055665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.086071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:43.087589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.179628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1759a-bf12-4a42-951a-7610994786e0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.194308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1050/2075] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:43.195680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.200542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.219397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e0be-2b96-4e28-ac61-67b62ef9f3ce None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.226770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.227492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.227844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.228119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.228883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.231238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.252185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:43.263247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.264483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.264951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.265412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] PATH_INFO: `/v3/projects/3e6c81f84dac4d9cafa3908a5896ae24/users/26c501f3361e46239a6a492d0858afe2/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.268674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1051/2076] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:43 2022] POST /identity/v3/users => generated 302 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:43.270910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.273445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.291965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6c81f84dac4d9cafa3908a5896ae24, user_id=26c501f3361e46239a6a492d0858afe2, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.295001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.295234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0402-c8e1-46ac-a25b-c32080e17faa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.310278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.310957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.311243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.311522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] PATH_INFO: `/v3/projects/707ec079cb184ae1a1cb248dbc64a4f0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.312497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1026/2077] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:43 2022] PUT /identity/v3/projects/3e6c81f84dac4d9cafa3908a5896ae24/users/26c501f3361e46239a6a492d0858afe2/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:43.313524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.315947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.316223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=707ec079cb184ae1a1cb248dbc64a4f0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.319955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.375279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:43.384814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707ec079cb184ae1a1cb248dbc64a4f0', 'request_id': 'req-52a5476e-1675-45b5-9101-b889086107ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.396590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707ec079cb184ae1a1cb248dbc64a4f0', 'request_id': 'req-52a5476e-1675-45b5-9101-b889086107ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.403635 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707ec079cb184ae1a1cb248dbc64a4f0', 'request_id': 'req-52a5476e-1675-45b5-9101-b889086107ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.409209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021e23-621e-43e3-b13c-945cfb73e047 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.416981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] The token cache is being invalidate because project 707ec079cb184ae1a1cb248dbc64a4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:43.416981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.418380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.423937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1052/2078] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:43 2022] DELETE /identity/v3/projects/707ec079cb184ae1a1cb248dbc64a4f0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:43.423937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.423937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.492469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5476e-1675-45b5-9101-b889086107ff admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.505369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.506997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.507288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.507483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.508646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.509179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.510041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.510755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.512401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.515625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86462f-9af3-45f4-a59f-e71fffdb6836 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.518444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.536460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1053/2079] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:43.537607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.539146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.539719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.540411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.541216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.555858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1027/2080] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:43.557408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.559590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.574404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:43.575171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500d745-d198-46e9-ba2b-f35ba7e32a31 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.575973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.580358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.581416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.581733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.582006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] PATH_INFO: `/v3/users/75d9b7fb705a4ecd9038177a3f9364bf` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.583236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d9b7fb705a4ecd9038177a3f9364bf)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.585179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.590601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1054/2081] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:43.591701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.605229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c4f1-27c6-4f81-8592-7abd77a5c855 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.619128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.620568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.621011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.621317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.622047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.624304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.641946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1055/2082] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:43.642812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.646889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.673836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d9b7fb705a4ecd9038177a3f9364bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.674548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d9b7fb705a4ecd9038177a3f9364bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.675394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6336387-abcb-43bc-a05e-5b4a47bf88a0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.682896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.683735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.684011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.684282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.687012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.692926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d9b7fb705a4ecd9038177a3f9364bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.721775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d9b7fb705a4ecd9038177a3f9364bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.725353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d9b7fb705a4ecd9038177a3f9364bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:43.746281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.751889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082e75-0737-44d7-945f-a9f6b0c960d8 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:43.770056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1028/2083] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:43 2022] DELETE /identity/v3/users/75d9b7fb705a4ecd9038177a3f9364bf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:43.770056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:43.770466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1056/2084] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:43.780841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:43.907718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307d3a5-7d45-46ca-b06f-f8329a8e394f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:43.919273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:43.920425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:43.920894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:43.921272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] PATH_INFO: `/v3/projects/1d68a465360b4c7aa88be7fa6d57d175/users/58fb9632873548ca9573d570ad86824e/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:43.942760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:43.986763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d68a465360b4c7aa88be7fa6d57d175, user_id=58fb9632873548ca9573d570ad86824e, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:43.989835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:44.033307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1029/2085] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:43 2022] PUT /identity/v3/projects/1d68a465360b4c7aa88be7fa6d57d175/users/58fb9632873548ca9573d570ad86824e/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:44.036303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.037034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.037772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.090771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:44.092478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.172291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:44.176568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.261458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:44.338222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1057/2086] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:44.374510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:44.380164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.415793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f342-a493-47ea-8f78-3e034be0c3fd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:44.428346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:44.429027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.429322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.429590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] PATH_INFO: `/v3/projects/61ea1dce6cf54ebd8d62c202f2aa0520` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.434142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ea1dce6cf54ebd8d62c202f2aa0520)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:44.436098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:44.518073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c3d40-e5d0-4d05-bac9-bbabfb646fe7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:44.534042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ea1dce6cf54ebd8d62c202f2aa0520', 'request_id': 'req-78f2ae3a-c573-4d01-b2f9-2944f47d1737'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.547046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ea1dce6cf54ebd8d62c202f2aa0520', 'request_id': 'req-78f2ae3a-c573-4d01-b2f9-2944f47d1737'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.559205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ea1dce6cf54ebd8d62c202f2aa0520', 'request_id': 'req-78f2ae3a-c573-4d01-b2f9-2944f47d1737'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.580616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] The token cache is being invalidate because project 61ea1dce6cf54ebd8d62c202f2aa0520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:44.581025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.581920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.583853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1030/2087] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:44 2022] DELETE /identity/v3/projects/61ea1dce6cf54ebd8d62c202f2aa0520 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:44.585067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:44.590152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.610360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:44.612048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.613007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.613824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.619333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.732169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ae3a-c573-4d01-b2f9-2944f47d1737 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:44.745149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:44.748567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646e0d-6f18-4fc7-9ea4-8455acb7a80a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:44.753057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:44.753823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.754129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.754472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] PATH_INFO: `/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.756191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b3dfab903f147dba042ebd4dbc0c8b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:44.759297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:44.798215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:44.820689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1058/2088] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:44.820689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ed42-e3ce-4d0e-9bb2-a078902def0e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.820689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ed42-e3ce-4d0e-9bb2-a078902def0e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.820689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ed42-e3ce-4d0e-9bb2-a078902def0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.845989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b3dfab903f147dba042ebd4dbc0c8b2', 'request_id': 'req-be9a7174-c319-4ff6-94ce-02edbdaae3bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.861785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b3dfab903f147dba042ebd4dbc0c8b2', 'request_id': 'req-be9a7174-c319-4ff6-94ce-02edbdaae3bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.867560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0370ed42-e3ce-4d0e-9bb2-a078902def0e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:44.869130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0370ed42-e3ce-4d0e-9bb2-a078902def0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.879584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b3dfab903f147dba042ebd4dbc0c8b2', 'request_id': 'req-be9a7174-c319-4ff6-94ce-02edbdaae3bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.913315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] The token cache is being invalidate because project 1b3dfab903f147dba042ebd4dbc0c8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:44.914035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.915008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-be9a7174-c319-4ff6-94ce-02edbdaae3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:44.920949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1031/2089] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:44 2022] DELETE /identity/v3/projects/1b3dfab903f147dba042ebd4dbc0c8b2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:44.920949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.920949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.920949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:44.950091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:44.955479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] MFA Rules not processed for user `26c501f3361e46239a6a492d0858afe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:44.956683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:44.979114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1059/2090] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:44.980520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2dbb-d027-4d8f-9b6f-cca40b14903c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:44.980892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2dbb-d027-4d8f-9b6f-cca40b14903c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:44.981468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2dbb-d027-4d8f-9b6f-cca40b14903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.009917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ebfc2dbb-d027-4d8f-9b6f-cca40b14903c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:45.011177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ebfc2dbb-d027-4d8f-9b6f-cca40b14903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:45.021431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1060/2091] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:45.022898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205c08-5d99-477f-ba04-87f8a8ff0642 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.023184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205c08-5d99-477f-ba04-87f8a8ff0642 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.023456 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205c08-5d99-477f-ba04-87f8a8ff0642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.081694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b205c08-5d99-477f-ba04-87f8a8ff0642 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:45.085009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b205c08-5d99-477f-ba04-87f8a8ff0642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:45.098977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1061/2092] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:45.103095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.103940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.104804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.124248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1032/2093] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:45.124248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.137268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:45.139905 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:45.144177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69d55a-e9df-44f1-97bb-f24722f5c0b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.151251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1062/2094] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:45.152603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.158782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:45.159828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.160482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.160763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.161588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.162689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:45.164007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.165729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.208513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1033/2095] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:45.208513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.216251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de032133-c637-4804-bf0c-c0c22df5bb5d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.221729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.222590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.222935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.223453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.224264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:45.225027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.226822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.249380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1034/2096] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:45.251023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.272762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde390a7-8e8f-4daf-ba53-49b3fc9b8b0a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.276921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7f4b6-2b94-49a9-a468-c8aa986e6613 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.282142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.282968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.283168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.283500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.283838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.284187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.284458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.284790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.286978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.289498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.299231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.302016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.318838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1063/2097] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:45.319801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.333780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1035/2098] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:45.338159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.345747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957940d-a743-4469-bff7-8d66193f7b26 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.418166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:45.438200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6948b8-632f-42fe-b373-da6d78c76669 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.447624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.449197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.449579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.449919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.450717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.452805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.485014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.485855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.486039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.486269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.486954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.488927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.505637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1036/2099] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:45.510421 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.532403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52a65-79e7-40bc-bd58-f0540dad79e8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.541288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1064/2100] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:45.543888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.545492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.548124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.548315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.548601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.549358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.554290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.580865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1037/2101] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:45.585392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.605299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b665e0-7cde-4a6c-9495-83f3c7a5167c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.697736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.699955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.700255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.700732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.701717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.703737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.705053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5ef2c-0498-47e2-aefb-5ebe5517ca02 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.719742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1038/2102] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:45.724850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.728469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.731469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.731912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.732314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.735272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.739108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.753727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:45.767154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086874ea-bc77-455a-80f1-09259b368d33 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.774016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.775081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.775711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.776196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] PATH_INFO: `/v3/projects/4a3590ecbf644a23b13e902cf2a22044/users/fd3917f5a04c42c0a1bba0cd1420fde9/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.803038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3590ecbf644a23b13e902cf2a22044, user_id=fd3917f5a04c42c0a1bba0cd1420fde9, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.816378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac954a8-fd5d-48af-b06b-0f676941ebb3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:45.824594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1065/2103] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:45.830033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:45.848247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1039/2104] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:45 2022] PUT /identity/v3/projects/4a3590ecbf644a23b13e902cf2a22044/users/fd3917f5a04c42c0a1bba0cd1420fde9/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:45.863966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.864253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.864539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.864911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b527a1-e1d7-440e-879a-9f348f7e6757 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:45.956296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] MFA Rules not processed for user `fd3917f5a04c42c0a1bba0cd1420fde9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:45.957688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:45.987603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:45.988257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:45.988535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:45.988808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:45.989724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:45.991361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.005383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1066/2105] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:46.009129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.032828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f99f9d-fc7f-4d3b-854a-ed1a47173a33 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.045953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.045953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.045953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.045953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.046710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.052729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd567f69-2c3d-44f6-b1a8-06d0777488be admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.078452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1067/2106] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:46.080715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662beed-a30c-415f-9a90-36e7a01952ee None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.081850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662beed-a30c-415f-9a90-36e7a01952ee None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.083177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662beed-a30c-415f-9a90-36e7a01952ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.100113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1040/2107] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:46.102273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.128178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262848a-bc0d-42b3-a004-d7d66678a66c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.132315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a662beed-a30c-415f-9a90-36e7a01952ee None None] MFA Rules not processed for user `58fb9632873548ca9573d570ad86824e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:46.134222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a662beed-a30c-415f-9a90-36e7a01952ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:46.139680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.140892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.141601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.142157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] PATH_INFO: `/v3/projects/bd252879187e4352902e9d8044463d92/users/d472a2b220f0412ba5c2b0f8b06fe839/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.166797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd252879187e4352902e9d8044463d92, user_id=d472a2b220f0412ba5c2b0f8b06fe839, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.172836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.201645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1041/2108] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:46 2022] PUT /identity/v3/projects/bd252879187e4352902e9d8044463d92/users/d472a2b220f0412ba5c2b0f8b06fe839/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:46.203163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.205459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:46.222353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:46.309203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1068/2109] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:46.317362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.317808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.318183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.363680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] MFA Rules not processed for user `d472a2b220f0412ba5c2b0f8b06fe839`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:46.365094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:46.368795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47f3d0-0bd0-4735-a60c-9472db6e0790 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.376829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.378048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.378568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.379017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.451887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.453517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.463752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:46.465876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1042/2110] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:46.467608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.475293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:46.497618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1069/2111] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:46.507725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.523694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91026d74-1d33-4def-9d9a-ce67b7273b80 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.537441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.543220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.544121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.544121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.574062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.581000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.598998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1070/2112] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:46.631429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7393c0-d6f1-49bf-937f-b3cd9912cc77 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.636519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.637080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.637408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.637652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.638328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.640229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.696452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1043/2113] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:46 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:46.701344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.717654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2854ca-1aa7-4691-a11b-5c4dbfdd48cf service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.804704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.806122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.806788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.808266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.810018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.812947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.848514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1071/2114] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:46 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:46.853801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.873625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67552f-57fc-43eb-ba1d-f0b1698abd1d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.878052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.878766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.879280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.879632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.880464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:46.882489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:46.899151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1044/2115] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:46.906112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:46.929843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf1f-9dcc-4316-aa00-4af40b37382b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:46.957215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:46.958376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:46.958996 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:46.959449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] PATH_INFO: `/v3/projects/89e4cd5d30ba445da3305ac0deb974fe/users/5a38234964b94530b64c14e6abc87409/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:46.998279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e4cd5d30ba445da3305ac0deb974fe, user_id=5a38234964b94530b64c14e6abc87409, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:47.006433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1486d3-e7eb-4ea1-889f-ca5cc1d7dbf9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:47.045646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1072/2116] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:46 2022] PUT /identity/v3/projects/89e4cd5d30ba445da3305ac0deb974fe/users/5a38234964b94530b64c14e6abc87409/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:47.046095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:47.086179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f75730-b7f7-40b1-8627-112880291406 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:47.095575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:47.097404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:47.098840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:47.164417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] MFA Rules not processed for user `26c501f3361e46239a6a492d0858afe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:47.166933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:47.178731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:47.262858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:47.263508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:47.263781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:47.266020 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:47.266833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:47.271753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:47.306874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1045/2117] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:47.311665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:47.335962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a99bf-1c9e-4e92-88ab-ec473923b5bd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:47.347703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:47.349792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:47.350092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:47.350495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] PATH_INFO: `/v3/projects/89e4cd5d30ba445da3305ac0deb974fe/users/5a38234964b94530b64c14e6abc87409/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:47.351277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1073/2118] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:47.360187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e4cd5d30ba445da3305ac0deb974fe, user_id=5a38234964b94530b64c14e6abc87409, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:47.363389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de32e9f-15a3-42cd-ae0d-3a574921b374 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:47.383457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1046/2119] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:47 2022] PUT /identity/v3/projects/89e4cd5d30ba445da3305ac0deb974fe/users/5a38234964b94530b64c14e6abc87409/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:48.235952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:48.238114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:48.281849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c089-bbf6-4102-b686-7838374ad5bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:48.509166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:48.562272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:48.566191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:48.568347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:48.568979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] PATH_INFO: `/v3/users/f7a97895b4a84483b9dcb4bc84592c41` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:48.572985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a97895b4a84483b9dcb4bc84592c41)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:48.580985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:48.720113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:48.721648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:48.722315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:48.738092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a97895b4a84483b9dcb4bc84592c41'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:48.770751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a97895b4a84483b9dcb4bc84592c41'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:48.771734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a97895b4a84483b9dcb4bc84592c41'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:48.772700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a97895b4a84483b9dcb4bc84592c41'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:48.793597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] MFA Rules not processed for user `5a38234964b94530b64c14e6abc87409`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:48.796218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:48.810578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a97895b4a84483b9dcb4bc84592c41'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:48.875030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1074/2120] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:48 2022] DELETE /identity/v3/users/f7a97895b4a84483b9dcb4bc84592c41 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:49.059999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1047/2121] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:49.375090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:49.377304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:49.587983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb7b16-a0a7-4867-9559-feddc567b887 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:49.598648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:49.599384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:49.599666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:49.599941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] PATH_INFO: `/v3/projects/bdda487459f9498982d40f0881da0540` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:49.602378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdda487459f9498982d40f0881da0540)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:49.606938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:49.734590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdda487459f9498982d40f0881da0540', 'request_id': 'req-25976b2f-5f77-4839-858d-1bd9599ef0ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.750266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdda487459f9498982d40f0881da0540', 'request_id': 'req-25976b2f-5f77-4839-858d-1bd9599ef0ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.761025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdda487459f9498982d40f0881da0540', 'request_id': 'req-25976b2f-5f77-4839-858d-1bd9599ef0ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.785362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] The token cache is being invalidate because project bdda487459f9498982d40f0881da0540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:49.786336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.787377 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.789569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1075/2122] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:49 2022] DELETE /identity/v3/projects/bdda487459f9498982d40f0881da0540 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:49.794268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:49.811953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8aa45-5606-41a1-b57c-6c6113fcdd21 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:49.886668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:49.887681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:49.888076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:49.888562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] PATH_INFO: `/v3/users/7cb9438be25f46709fefce8523e6caae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:49.890801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb9438be25f46709fefce8523e6caae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:49.895109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:49.975742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:49.983166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb9438be25f46709fefce8523e6caae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.984497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb9438be25f46709fefce8523e6caae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:49.997348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb9438be25f46709fefce8523e6caae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.010096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb9438be25f46709fefce8523e6caae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.011901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7cd60cf5-d93c-450e-a3a6-4c6e03c94597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb9438be25f46709fefce8523e6caae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.046603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1048/2123] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:49 2022] DELETE /identity/v3/users/7cb9438be25f46709fefce8523e6caae => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:50.392673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:50.396140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:50.408741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:50.429682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25976b2f-5f77-4839-858d-1bd9599ef0ce admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:50.471435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774d82-694b-403b-a9d4-e53930c9a4d4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:50.471710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774d82-694b-403b-a9d4-e53930c9a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:50.471868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774d82-694b-403b-a9d4-e53930c9a4d4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:50.473815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1049/2124] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:44:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 22:44:50.510859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:50.510859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:50.510859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:50.555607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:50.556266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:50.556599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:50.556948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] PATH_INFO: `/v3/projects/940c487056854d8386576adc53e02956` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:50.559085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940c487056854d8386576adc53e02956)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:50.561483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:50.578196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] MFA Rules not processed for user `30e6ac9cc1264e109a68bc20fd975623`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:50.581813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:50.638886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940c487056854d8386576adc53e02956', 'request_id': 'req-ff14d270-0fd8-45db-844f-aa059316ed51'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.653682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940c487056854d8386576adc53e02956', 'request_id': 'req-ff14d270-0fd8-45db-844f-aa059316ed51'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.673748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940c487056854d8386576adc53e02956', 'request_id': 'req-ff14d270-0fd8-45db-844f-aa059316ed51'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.705344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] The token cache is being invalidate because project 940c487056854d8386576adc53e02956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:50.705864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.706967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff14d270-0fd8-45db-844f-aa059316ed51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:50.710687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1076/2125] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:50 2022] DELETE /identity/v3/projects/940c487056854d8386576adc53e02956 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:50.722541 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:50.723045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:50.723291 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:50.755536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:50.757582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:50.762260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1050/2126] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:44:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:50.797612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:50.799579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:50.829813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631208e-932d-456f-abd4-37d9e25ea9ed None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:50.882810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1077/2127] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:50.898036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:50.900363 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:50.975543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:51.035389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b14dcb-747c-4f8d-bd89-4cad47332b00 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.044352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.046513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.047235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.047701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.091749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.094833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab748226-4de2-43b8-b149-21fddadbdb3d service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.116489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.118480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.119209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.119951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] PATH_INFO: `/v3/users/fd3917f5a04c42c0a1bba0cd1420fde9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.121641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1078/2128] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:51.123273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.123641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.123982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.125455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3917f5a04c42c0a1bba0cd1420fde9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.129516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.157220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:51.160336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.185781 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1079/2129] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:51.188449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.225895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc521f-504d-4d6b-90d7-ef422b10fb07 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.239116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.241594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.242308 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.242727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.243338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3917f5a04c42c0a1bba0cd1420fde9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:51.244236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:51.244742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.245409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3917f5a04c42c0a1bba0cd1420fde9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:51.251029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.272221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3917f5a04c42c0a1bba0cd1420fde9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:51.272482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1080/2130] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:51.279117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.282093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:51.282558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.293792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3917f5a04c42c0a1bba0cd1420fde9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:51.295654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c347abdc-465f-498d-b3a5-8ff2e68365d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3917f5a04c42c0a1bba0cd1420fde9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:51.308621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a312-b8ec-46eb-ab4b-007dcd68a094 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.316712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.318295 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.318614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.318986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.320526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.322825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.341667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1051/2131] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:50 2022] DELETE /identity/v3/users/fd3917f5a04c42c0a1bba0cd1420fde9 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:51.347649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.348258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.348823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.372380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1081/2132] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:51 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:51.378712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.387564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:51.391227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.473719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf813c-647f-4e12-a864-f525f22e1702 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.480515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.481653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.482300 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.482722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.484002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.486820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.516216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1052/2133] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:51.520611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1082/2134] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:51 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:51.527157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.536484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.542221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.547741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b81df-29af-4e67-9c54-2bce0550dce0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.556323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.557601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.558165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.558709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.560540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.563672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.593665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1053/2135] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:51.598655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.618482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2f53-1576-4c25-a34d-6829f918b607 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.638389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.639002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15ffa-a57e-4ce9-bed7-204b2983d3a7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.639179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.639357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.639592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.641370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.643085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.645809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.647083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.647569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.647908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.656178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1054/2136] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:51.660136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.672062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.673707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc952f-4e99-4395-9f0d-c2861ef7a0b3 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.683528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698c71-0ce3-4802-9c4c-1b317263d3ac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:51.689073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1083/2137] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:51.696692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:51.698008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.698638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.699298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] PATH_INFO: `/v3/projects/5bc61ee6b1ad4ad1a6e64810a6e282d5/users/b544405861c64c1fbab8707e3b1e2151/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.727747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc61ee6b1ad4ad1a6e64810a6e282d5, user_id=b544405861c64c1fbab8707e3b1e2151, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:51.733535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:51.766288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1055/2138] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:51 2022] PUT /identity/v3/projects/5bc61ee6b1ad4ad1a6e64810a6e282d5/users/b544405861c64c1fbab8707e3b1e2151/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:51.773584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:51.774101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:51.774611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:51.823388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:51.836385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] MFA Rules not processed for user `b544405861c64c1fbab8707e3b1e2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:51.838944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.887210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.888937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:51.981191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1084/2139] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:51.993723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:51.997815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:52.002560 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f5ad-455a-459e-b205-aa7f5af68692 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:52.018380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:52.018982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:52.019265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:52.019594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] PATH_INFO: `/v3/users/58fb9632873548ca9573d570ad86824e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:52.022889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] RBAC: Authorizing `identity:delete_user(user_id=58fb9632873548ca9573d570ad86824e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:52.025037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:52.070776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:52.112608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859b7e-6a27-48c2-91bb-a1962d65409d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:52.117309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:52.117997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:52.118359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:52.119910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:52.128175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58fb9632873548ca9573d570ad86824e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:52.130702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58fb9632873548ca9573d570ad86824e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:52.131992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:52.137791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:52.156320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58fb9632873548ca9573d570ad86824e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:52.168833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1085/2140] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:52.174885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58fb9632873548ca9573d570ad86824e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:52.175804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58fb9632873548ca9573d570ad86824e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:52.241875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1056/2141] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:51 2022] DELETE /identity/v3/users/58fb9632873548ca9573d570ad86824e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:52.731670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:52.736442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:52.787858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4317b-0d5a-4cf6-a3b8-bb4fecadc112 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:52.916285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:52.917462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:52.917953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:52.918397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] PATH_INFO: `/v3/projects/4a3590ecbf644a23b13e902cf2a22044` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:52.920786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3590ecbf644a23b13e902cf2a22044)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:52.924165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:52.948204 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:52.950807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:52.974165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a07ef2-aba4-4c8e-8305-1a2440d6539d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:52.982600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:52.983652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:52.984393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:52.985100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] PATH_INFO: `/v3/projects/1d68a465360b4c7aa88be7fa6d57d175` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:52.987333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d68a465360b4c7aa88be7fa6d57d175)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:52.991523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.041801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:53.043768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3590ecbf644a23b13e902cf2a22044', 'request_id': 'req-5cf36149-e3bf-4f3c-b542-2e146a0014cd'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.054674 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3590ecbf644a23b13e902cf2a22044', 'request_id': 'req-5cf36149-e3bf-4f3c-b542-2e146a0014cd'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.066127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3590ecbf644a23b13e902cf2a22044', 'request_id': 'req-5cf36149-e3bf-4f3c-b542-2e146a0014cd'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.083129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] The token cache is being invalidate because project 4a3590ecbf644a23b13e902cf2a22044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:53.083960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.086193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5cf36149-e3bf-4f3c-b542-2e146a0014cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.088707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1086/2142] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:52 2022] DELETE /identity/v3/projects/4a3590ecbf644a23b13e902cf2a22044 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:53.100763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d5da-9552-4ef6-aa42-271274e555ff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.101003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d68a465360b4c7aa88be7fa6d57d175', 'request_id': 'req-668fb77b-f923-49d1-ade5-c6cff1dd6da3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.101165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d5da-9552-4ef6-aa42-271274e555ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.101165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d5da-9552-4ef6-aa42-271274e555ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.123362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1ac3d5da-9552-4ef6-aa42-271274e555ff None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:53.125072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1ac3d5da-9552-4ef6-aa42-271274e555ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:53.136430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d68a465360b4c7aa88be7fa6d57d175', 'request_id': 'req-668fb77b-f923-49d1-ade5-c6cff1dd6da3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.156328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d68a465360b4c7aa88be7fa6d57d175', 'request_id': 'req-668fb77b-f923-49d1-ade5-c6cff1dd6da3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.185287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] The token cache is being invalidate because project 1d68a465360b4c7aa88be7fa6d57d175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:53.185897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.188453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.191896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1057/2143] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:52 2022] DELETE /identity/v3/projects/1d68a465360b4c7aa88be7fa6d57d175 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:53.199756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.202566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:53.215711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1087/2144] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:53.227027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.227385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.227744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.242766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fb77b-f923-49d1-ade5-c6cff1dd6da3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.256405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:53.258246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:53.269740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.272926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1088/2145] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:53.274002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.274342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.274631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] PATH_INFO: `/v3/users/5a38234964b94530b64c14e6abc87409` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.279554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.283138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a38234964b94530b64c14e6abc87409)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.287098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.297962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fec27-c160-41aa-afb2-4b9fcbb1d905 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.308276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.310864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.311967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.312912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.314312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:53.314807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.317892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.345400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1089/2146] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:53.352261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.371075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5290ca-2b93-4450-85a8-6d49f23dcaaf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.373550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a38234964b94530b64c14e6abc87409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.374811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a38234964b94530b64c14e6abc87409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.380271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.380916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.381196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.381449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.382172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.384088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.391022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a38234964b94530b64c14e6abc87409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.412806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a38234964b94530b64c14e6abc87409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.413873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a38234964b94530b64c14e6abc87409'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:53.428818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1090/2147] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:53 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:53.433959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.438526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:53.459119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1058/2148] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:53 2022] DELETE /identity/v3/users/5a38234964b94530b64c14e6abc87409 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:53.548616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a35397-0242-4a08-8633-df977ecbbace admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.617685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:53.666298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.667214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.667334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.667551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.668217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.672384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.716499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1091/2149] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:53 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:53.720069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.732360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:53.740412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59293218-a36e-44b6-b7a7-f98b1c706d23 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.754817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.755677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.756094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.756481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.757411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.765541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.786396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1059/2150] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:53.790691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.806894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfccdfe-74b5-450f-911e-a2de2d0c84d1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.816968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.824405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:53.843376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.844132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd446a-bcd6-48af-ba71-8440bfea3d1f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.845978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.846835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.847628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.851411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.856810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.860493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.873333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.873822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.874115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] PATH_INFO: `/v3/projects/89e4cd5d30ba445da3305ac0deb974fe` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.880208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e4cd5d30ba445da3305ac0deb974fe)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.882228 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:53.885488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1092/2151] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:53.891725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:53.934207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584253d-84e0-4536-b8bf-2e5a30b88562 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:53.948594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:53.949590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:53.950427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:53.950857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] PATH_INFO: `/v3/projects/9e89f6a32366452592eb5a76f6af57fe/users/c3c864261ae94460ad72e2c3b709d8e3/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:53.977091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e89f6a32366452592eb5a76f6af57fe, user_id=c3c864261ae94460ad72e2c3b709d8e3, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:53.982628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9228db8-a406-412c-938a-144f9ff03256 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.003985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e4cd5d30ba445da3305ac0deb974fe', 'request_id': 'req-525be62d-dd06-46a6-89b9-6dfc5324837a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:54.030382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1093/2152] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:53 2022] PUT /identity/v3/projects/9e89f6a32366452592eb5a76f6af57fe/users/c3c864261ae94460ad72e2c3b709d8e3/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:54.030382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.030382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.030902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.031884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e4cd5d30ba445da3305ac0deb974fe', 'request_id': 'req-525be62d-dd06-46a6-89b9-6dfc5324837a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:54.054199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e4cd5d30ba445da3305ac0deb974fe', 'request_id': 'req-525be62d-dd06-46a6-89b9-6dfc5324837a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:54.097297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] MFA Rules not processed for user `c3c864261ae94460ad72e2c3b709d8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:54.100564 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:54.133167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] The token cache is being invalidate because project 89e4cd5d30ba445da3305ac0deb974fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:54.143270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:54.150298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-525be62d-dd06-46a6-89b9-6dfc5324837a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:54.154066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1060/2153] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:53 2022] DELETE /identity/v3/projects/89e4cd5d30ba445da3305ac0deb974fe => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:54.162513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8916-e826-491a-8afb-8d6e04047727 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.163721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8916-e826-491a-8afb-8d6e04047727 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.164187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce8916-e826-491a-8afb-8d6e04047727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.205400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dbce8916-e826-491a-8afb-8d6e04047727 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:54.208245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dbce8916-e826-491a-8afb-8d6e04047727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:54.242487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1094/2154] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:54.264154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.267273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:54.360783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1061/2155] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:54.373901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.373901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.374200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.388850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5900f15-9293-4c80-a831-9c5741228ed5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.394013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.394758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.395128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.395763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.412718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.416289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.424760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:54.426819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:54.434649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1095/2156] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:54.445845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1062/2157] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:54.450938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.466657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8132acd-3784-47d7-85cd-fa5af52d56d6 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.472945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.473730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.474171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.474652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.475866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:54.476432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.479021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.498091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1096/2158] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:44:54.503000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.513193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7056-5263-453f-95a0-59340a1388e5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.518330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.518976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.519255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.519532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.520342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.523863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.562820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1063/2159] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:54.568718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.582575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2111e-0114-4dcb-a2da-b2c7f5489851 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.603647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:54.695836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.697258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.698071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.698728 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.699895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.703844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.745810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1097/2160] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:54.747686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.764555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dbf5f-ec25-4756-8acf-00d0bc3a98cc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.777669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.777669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.777669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.777669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.778854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.781466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.796206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1064/2161] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:54.801914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.814752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e002e-f135-464f-87e0-f99d64b7f7cf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.820158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.821164 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.821676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.822033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.822979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.824678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648acd82-0f2e-40ad-bda0-0221bc8cb6bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.837381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1098/2162] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:54.845115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:54.855408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270108-2798-41ff-bc60-0a62193ba2fb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:54.860418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:54.862151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.862151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.862151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] PATH_INFO: `/v3/projects/0f535366072947da9a2547ce3f232f5a/users/f66f27b197fc4e4d91312e31c3bb1acb/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.886820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f535366072947da9a2547ce3f232f5a, user_id=f66f27b197fc4e4d91312e31c3bb1acb, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:54.891940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:54.917837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1065/2163] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:54 2022] PUT /identity/v3/projects/0f535366072947da9a2547ce3f232f5a/users/f66f27b197fc4e4d91312e31c3bb1acb/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:54.929119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:54.929408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:54.929720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:54.961521 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] MFA Rules not processed for user `f66f27b197fc4e4d91312e31c3bb1acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:54.962928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:55.039662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1099/2164] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:55.044604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.060545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:55.118583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361703a-952a-419d-8b18-f6098c7156c6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.123608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.124237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.124501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.124766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.125456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.127402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.179072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1066/2165] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:55 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:55.180515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.195213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee6e90-1912-4b21-93d7-3c7f83f406e2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.289233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.290372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.290775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.291224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.292362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.295343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.323715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1100/2166] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:55 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:55.328144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.341623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6099-5b5c-49a6-af09-f16566d9ab8b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.362517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.364260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.364608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.365744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.366608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.368608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.383766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1067/2167] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:55.388325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.399348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fb922-fbd5-49bc-ac57-967ee26dd87c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.408580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.409290 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.409591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.409880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] PATH_INFO: `/v3/projects/60cf4264f0744060b8e0a20f9a040b59/users/f7fbd1c6380d458c8e087aaf0d2b8e32/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.427231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:55.432316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60cf4264f0744060b8e0a20f9a040b59, user_id=f7fbd1c6380d458c8e087aaf0d2b8e32, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.435661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.482811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1101/2168] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:55 2022] PUT /identity/v3/projects/60cf4264f0744060b8e0a20f9a040b59/users/f7fbd1c6380d458c8e087aaf0d2b8e32/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:55.487001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.498284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a021f05-4e42-45d4-aa1a-0fea2b230ad0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.595645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.596849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.597312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.597880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.599758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.606212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498353e-fe54-42fb-9135-831ca80f7d18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.617792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1068/2169] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:55.622203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.633927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5812374-94f6-4c9c-82e9-b5ab2ef53718 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.642162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.643599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.643883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.644162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] PATH_INFO: `/v3/projects/60cf4264f0744060b8e0a20f9a040b59/users/f7fbd1c6380d458c8e087aaf0d2b8e32/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.646469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=60cf4264f0744060b8e0a20f9a040b59, user_id=f7fbd1c6380d458c8e087aaf0d2b8e32, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.649249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.664255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1102/2170] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:55 2022] PUT /identity/v3/projects/60cf4264f0744060b8e0a20f9a040b59/users/f7fbd1c6380d458c8e087aaf0d2b8e32/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:55.676745 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.677018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.677287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.710627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] MFA Rules not processed for user `f7fbd1c6380d458c8e087aaf0d2b8e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:55.712063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:55.789388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1069/2171] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:55.802449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.804436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:55.871333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9fe0-4e32-472f-9f13-2feaaff907bf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:55.875134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:55.875877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:55.876182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:55.876450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:55.886661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:55.888109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f870c72-38fc-4057-95ff-c9a567b6927f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:55.898593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1103/2172] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:55.934082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:55.947094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:56.009128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691277d9-45b3-4ad9-a480-b268c2cf5b59 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:56.019712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:56.020542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:56.020843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:56.021120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] PATH_INFO: `/v3/users/f66f27b197fc4e4d91312e31c3bb1acb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:56.022572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66f27b197fc4e4d91312e31c3bb1acb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:56.024566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:56.078686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66f27b197fc4e4d91312e31c3bb1acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.079409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66f27b197fc4e4d91312e31c3bb1acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.086997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66f27b197fc4e4d91312e31c3bb1acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.097169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66f27b197fc4e4d91312e31c3bb1acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.098155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66f27b197fc4e4d91312e31c3bb1acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.117955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1070/2173] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:55 2022] DELETE /identity/v3/users/f66f27b197fc4e4d91312e31c3bb1acb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:56.123102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:56.123392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:56.123661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:56.144586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:56.145838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:56.212821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1104/2174] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:56.226376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:56.227930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:56.299681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f24ad-8b1a-4d05-b78b-558b2ab3ab46 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:56.304769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:56.305455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:56.305766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:56.306049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:56.317097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:56.318689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:56.330289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1071/2175] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:56.800450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:56.803222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:56.818385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b4bca-5399-43a6-b472-214235345b17 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:56.823501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:56.824178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:56.824532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:56.824820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] PATH_INFO: `/v3/projects/0f535366072947da9a2547ce3f232f5a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:56.826204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f535366072947da9a2547ce3f232f5a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:56.828185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:56.884399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f535366072947da9a2547ce3f232f5a', 'request_id': 'req-458ea219-e662-4407-8274-cbdb289d3a64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.892864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f535366072947da9a2547ce3f232f5a', 'request_id': 'req-458ea219-e662-4407-8274-cbdb289d3a64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.899365 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f535366072947da9a2547ce3f232f5a', 'request_id': 'req-458ea219-e662-4407-8274-cbdb289d3a64'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.912024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] The token cache is being invalidate because project 0f535366072947da9a2547ce3f232f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:56.912317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.912975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:56.914555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1105/2176] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:56 2022] DELETE /identity/v3/projects/0f535366072947da9a2547ce3f232f5a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:56.918832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:56.929607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1593cb-5415-48ee-af9e-bbd5d8eb4312 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:56.965420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:57.010134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:57.010619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.010891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.011164 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] PATH_INFO: `/v3/users/f7fbd1c6380d458c8e087aaf0d2b8e32` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.012527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fbd1c6380d458c8e087aaf0d2b8e32)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:57.014735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:57.073936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fbd1c6380d458c8e087aaf0d2b8e32'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.074618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fbd1c6380d458c8e087aaf0d2b8e32'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.082025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fbd1c6380d458c8e087aaf0d2b8e32'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.091830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fbd1c6380d458c8e087aaf0d2b8e32'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.092474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57471ded-f0ff-44cd-907e-646ef2bda72d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fbd1c6380d458c8e087aaf0d2b8e32'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.111886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1072/2177] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:56 2022] DELETE /identity/v3/users/f7fbd1c6380d458c8e087aaf0d2b8e32 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:57.411223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:57.412727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:57.423137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:57.431887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ea219-e662-4407-8274-cbdb289d3a64 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:57.500692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:57.501370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.501680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.501971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] PATH_INFO: `/v3/projects/60cf4264f0744060b8e0a20f9a040b59` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.503412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60cf4264f0744060b8e0a20f9a040b59)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:57.505520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:57.580962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60cf4264f0744060b8e0a20f9a040b59', 'request_id': 'req-9b0dcfde-970c-4388-8577-7a106bbc4f71'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.590565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60cf4264f0744060b8e0a20f9a040b59', 'request_id': 'req-9b0dcfde-970c-4388-8577-7a106bbc4f71'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.599673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60cf4264f0744060b8e0a20f9a040b59', 'request_id': 'req-9b0dcfde-970c-4388-8577-7a106bbc4f71'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.614942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] The token cache is being invalidate because project 60cf4264f0744060b8e0a20f9a040b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:44:57.615251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.615910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9b0dcfde-970c-4388-8577-7a106bbc4f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:57.617431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1106/2178] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:57 2022] DELETE /identity/v3/projects/60cf4264f0744060b8e0a20f9a040b59 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:57.623840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.624523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.625184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.647408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:57.648690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:57.757414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1073/2179] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:57.767213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.767495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.767754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.802196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:57.803577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:57.812607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1107/2180] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:57.818376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:57.837270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe036a8-86a7-4519-9dfd-94b27470231c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:57.844021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:57.844687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.845006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.845233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.847272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:44:57.847272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:57.848339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:57.869226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1074/2181] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:44:57.873692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:57.885609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb4130-3d97-4392-9051-18c90f8d42b7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:57.896426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:57.897425 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:57.898014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:57.898478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:57.900216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:57.903019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96dfcc-1305-4988-adb6-44d593903fd6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:57.953857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1108/2182] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:57 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:57.959451 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:57.975818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fbff1-4641-4000-8bc6-fa83c67c2cc3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.038697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:58.084568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.085592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.085592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.085871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.086243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be42d3-8938-4799-bf08-5943202c4e72 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.086568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.086725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be42d3-8938-4799-bf08-5943202c4e72 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.087094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be42d3-8938-4799-bf08-5943202c4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.088436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.112787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e9be42d3-8938-4799-bf08-5943202c4e72 None None] MFA Rules not processed for user `c3c864261ae94460ad72e2c3b709d8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:58.114830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9be42d3-8938-4799-bf08-5943202c4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:58.115835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1075/2183] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:57 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:58.121183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.133939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ea5e0-bedd-4a47-ac1c-5d47be05545c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.139512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.140149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.140453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.140712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.141429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.143281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.154138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1076/2184] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:58.158489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.169300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12caad-8b7d-462f-9d7b-8c309a8b78d4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.174466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.175102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.175386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.175663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.176376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.178060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.188879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1077/2185] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:44:58.189593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1109/2186] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:44:58.194096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.210723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da143f2-d5ce-4acf-b492-d52c5c1dfe37 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.216306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.217026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.217324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.217653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] PATH_INFO: `/v3/projects/9a5ec3a58213496e82beca7982dbec25/users/33e7469c93da47f99cc2112831b6fd0f/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.233615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a5ec3a58213496e82beca7982dbec25, user_id=33e7469c93da47f99cc2112831b6fd0f, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.236511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.252170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1078/2187] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:58 2022] PUT /identity/v3/projects/9a5ec3a58213496e82beca7982dbec25/users/33e7469c93da47f99cc2112831b6fd0f/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:58.264005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.264297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.264569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.285609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:58.296483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] MFA Rules not processed for user `33e7469c93da47f99cc2112831b6fd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:58.298369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:58.371793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1110/2188] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:58.376716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.473180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3b809-80f6-4ba4-b751-370f33719c18 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.481197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.482291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.482754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.483214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.484461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.487639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.528502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1079/2189] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:44:58 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:44:58.533439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.545095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b75a65-045e-4dd1-8ee5-abbe60a3ea44 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.617837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.618552 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.619018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.619292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.620071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.622366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.650049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1111/2190] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:44:58 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:44:58.654291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.665718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ac41-ab70-41e6-ab93-d81921face75 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.672648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.673454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.673853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.674213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.675067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.676866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.692215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1080/2191] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:58.696779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.707439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb41f7-d15e-4f90-9865-77452e422f37 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.713470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.714194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.714471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.714743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] PATH_INFO: `/v3/projects/41e7c150072d4bfca7078c9f87b67b01/users/80673b0f404d4f248e16a7bed24297b2/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.733960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e7c150072d4bfca7078c9f87b67b01, user_id=80673b0f404d4f248e16a7bed24297b2, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.745233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.763488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.764034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1112/2192] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:58 2022] PUT /identity/v3/projects/41e7c150072d4bfca7078c9f87b67b01/users/80673b0f404d4f248e16a7bed24297b2/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:44:58.774748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec18f-d39d-4d04-bb09-a631eab953f4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.871078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:58.879266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.882104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.882406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.882698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.883493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.885158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f1505-dd5e-4ef6-b73e-9918c3dd4f88 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.896145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1081/2193] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:44:58.900580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:58.920724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50d960-c0a1-40ad-8ef3-854584f47a5f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:58.933725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:58.934442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.934722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.935000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] PATH_INFO: `/v3/projects/41e7c150072d4bfca7078c9f87b67b01/users/80673b0f404d4f248e16a7bed24297b2/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:58.937083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e7c150072d4bfca7078c9f87b67b01, user_id=80673b0f404d4f248e16a7bed24297b2, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:58.939813 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:58.952879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1113/2194] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:44:58 2022] PUT /identity/v3/projects/41e7c150072d4bfca7078c9f87b67b01/users/80673b0f404d4f248e16a7bed24297b2/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:58.964038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:58.964320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:58.964651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:59.007684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] MFA Rules not processed for user `80673b0f404d4f248e16a7bed24297b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:59.010599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:59.115737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1082/2195] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:59.129380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:59.131625 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:59.207914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ccb5c-3fe7-4f21-aab1-2959f54f3b1d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:59.213006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:59.214069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:59.214577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:59.215347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:59.222152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:59.226724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:59.228925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752941a-16e8-41c8-a1a7-e84b0857b31f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:59.243050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1114/2196] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:44:59.311649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:59.375840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c35787-69e8-4490-97dc-56f62517dd33 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:59.380930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:59.381595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:59.382021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:59.382300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] PATH_INFO: `/v3/users/33e7469c93da47f99cc2112831b6fd0f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:59.383541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e7469c93da47f99cc2112831b6fd0f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:59.385595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:59.438664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e7469c93da47f99cc2112831b6fd0f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:59.439520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e7469c93da47f99cc2112831b6fd0f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:59.447381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e7469c93da47f99cc2112831b6fd0f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:59.456840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e7469c93da47f99cc2112831b6fd0f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:59.457498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e7469c93da47f99cc2112831b6fd0f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:44:59.469854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:44:59.478826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1083/2197] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:44:59 2022] DELETE /identity/v3/users/33e7469c93da47f99cc2112831b6fd0f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:44:59.484323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:59.484487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:59.484752 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:59.504685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:44:59.505841 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:59.570947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1115/2198] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:44:59.581057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:59.582611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:59.643698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe86c63-de92-4f5e-9d1f-942676c7ef1b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:44:59.647693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:44:59.648319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:44:59.648593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:44:59.648846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:44:59.659160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:44:59.660605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:44:59.671043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1084/2199] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:44:59.982982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:44:59.984660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:44:59.997609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb43b2-65f2-46d0-96e5-8060fafa9e6b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:00.002372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:00.003394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:00.003394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:00.003572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] PATH_INFO: `/v3/projects/9a5ec3a58213496e82beca7982dbec25` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:00.006113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a5ec3a58213496e82beca7982dbec25)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:00.008432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:00.057629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a5ec3a58213496e82beca7982dbec25', 'request_id': 'req-2d3e3c2c-421d-4128-a770-3ffd607067c3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.066127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a5ec3a58213496e82beca7982dbec25', 'request_id': 'req-2d3e3c2c-421d-4128-a770-3ffd607067c3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.072339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a5ec3a58213496e82beca7982dbec25', 'request_id': 'req-2d3e3c2c-421d-4128-a770-3ffd607067c3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.084562 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] The token cache is being invalidate because project 9a5ec3a58213496e82beca7982dbec25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:00.084882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.085597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.087326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1116/2200] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:44:59 2022] DELETE /identity/v3/projects/9a5ec3a58213496e82beca7982dbec25 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:45:00.092178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:00.102807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5612f-7a29-4b63-a239-7e2d3bb38928 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:00.172200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:00.173047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:00.173256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:00.173531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] PATH_INFO: `/v3/users/80673b0f404d4f248e16a7bed24297b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:00.174824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80673b0f404d4f248e16a7bed24297b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:00.176828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:00.229969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80673b0f404d4f248e16a7bed24297b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.230704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80673b0f404d4f248e16a7bed24297b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.239098 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80673b0f404d4f248e16a7bed24297b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.248487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80673b0f404d4f248e16a7bed24297b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.249134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6df1d46e-54a3-4028-9afe-c2c4a7fafc12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80673b0f404d4f248e16a7bed24297b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.274186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1085/2201] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:00 2022] DELETE /identity/v3/users/80673b0f404d4f248e16a7bed24297b2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:00.673954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:00.675549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:00.689867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e3c2c-421d-4128-a770-3ffd607067c3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:00.751262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:00.751929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:00.752207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:00.752463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] PATH_INFO: `/v3/projects/41e7c150072d4bfca7078c9f87b67b01` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:00.753724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e7c150072d4bfca7078c9f87b67b01)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:00.755697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:00.777942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:00.808626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e7c150072d4bfca7078c9f87b67b01', 'request_id': 'req-1e8dd106-abe7-4658-b19e-cc43a6276cd1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.820314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e7c150072d4bfca7078c9f87b67b01', 'request_id': 'req-1e8dd106-abe7-4658-b19e-cc43a6276cd1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.826942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e7c150072d4bfca7078c9f87b67b01', 'request_id': 'req-1e8dd106-abe7-4658-b19e-cc43a6276cd1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.838555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] The token cache is being invalidate because project 41e7c150072d4bfca7078c9f87b67b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:00.838837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.839538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1e8dd106-abe7-4658-b19e-cc43a6276cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:00.841399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1117/2202] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:00 2022] DELETE /identity/v3/projects/41e7c150072d4bfca7078c9f87b67b01 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:00.846286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:00.846556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:00.846835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:00.875276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:00.877419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:00.896669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:00.979933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1086/2203] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:00.991403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:00.992090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:00.992090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.027064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:01.028997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:01.044337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1118/2204] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:01.049809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.066175 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed153a-e4a9-4ece-b640-d812cde0a947 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.073171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.074289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.074748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.075249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.076709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:01.077220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.080594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.105799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1087/2205] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:01.109336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.128450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294509b-706f-46de-a0cc-e3bc9f14df6e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.139899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.141133 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.141759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.143102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.143643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.148861 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.190448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1119/2206] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:01 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:01.195538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.207721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89beb7-d7a2-472a-9fb1-dccbc70bcbe8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.319850 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.321057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.321522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.322065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.323448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.326949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.362860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1088/2207] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:01 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:01.364217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.379232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b06fc-5967-4f32-a9a0-cbe0c15e21ac admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.386505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.387152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.387479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.387697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.388415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.390067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.402698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1120/2208] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:01.406814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.417195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883075-098c-402f-a6ac-bbb880c3b22e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.421900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.422532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.422807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.423073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.423844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.425462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3092b18-79bb-45e1-a38a-2cf9c41239de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.438443 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1089/2209] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:01.442660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.455962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7c2c-675d-447a-b74b-f17a3e35be7b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.461901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.462698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.463031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.463317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] PATH_INFO: `/v3/projects/ec22326a930d48b99f08f4826efc5f4a/users/e5d72758601b46359914918c442dc767/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.482873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec22326a930d48b99f08f4826efc5f4a, user_id=e5d72758601b46359914918c442dc767, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.486091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.526733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1121/2210] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:01 2022] PUT /identity/v3/projects/ec22326a930d48b99f08f4826efc5f4a/users/e5d72758601b46359914918c442dc767/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:01.538687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.538880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.539285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.569294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] MFA Rules not processed for user `e5d72758601b46359914918c442dc767`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:01.570394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:01.648334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1090/2211] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:01.653112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.725921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67372fb8-3155-4a9f-855a-e6ca8b137563 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.731197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.731832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.732112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.732385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.733073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.735045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.765765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1122/2212] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:01 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:01.769940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.782016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09e487-d8d4-471b-beac-a69924dc19e6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.793465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:01.848814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.849388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.849717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.849993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.850685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.852648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.879397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1091/2213] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:01 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:45:01.884106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.897035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f93b-4e1d-4223-b9e2-700ebef83ccb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.904134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.904897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.905176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.905426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.906179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.907838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.920716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1123/2214] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:01.925102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:01.944241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41232ad3-d6ad-4d81-b422-638ba3f8e6ef admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:01.954106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:01.954858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:01.955139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:01.955400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] PATH_INFO: `/v3/projects/1a19aa17d0b14412a4d696f14ea643b6/users/aa0b5510413b4ee6b1dd0048d3df7c06/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:01.974183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a19aa17d0b14412a4d696f14ea643b6, user_id=aa0b5510413b4ee6b1dd0048d3df7c06, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:01.976846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:01.992824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1092/2215] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:01 2022] PUT /identity/v3/projects/1a19aa17d0b14412a4d696f14ea643b6/users/aa0b5510413b4ee6b1dd0048d3df7c06/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:01.997209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:02.010405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31dba7-980a-4df8-87f0-0f1e3957fcf2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:02.024889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:02.076622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:02.077256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:02.077542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:02.077846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:02.078555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:02.080150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2595794-ebc2-40cd-9bd4-c3abcc1f85ec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:02.091809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1124/2216] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:02.096261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:02.107463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d420a76-830b-40e6-8292-092906395dd7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:02.112181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:02.112854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:02.113127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:02.113394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] PATH_INFO: `/v3/projects/1a19aa17d0b14412a4d696f14ea643b6/users/aa0b5510413b4ee6b1dd0048d3df7c06/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:02.115378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a19aa17d0b14412a4d696f14ea643b6, user_id=aa0b5510413b4ee6b1dd0048d3df7c06, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:02.118084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:02.131188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1093/2217] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:02 2022] PUT /identity/v3/projects/1a19aa17d0b14412a4d696f14ea643b6/users/aa0b5510413b4ee6b1dd0048d3df7c06/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:02.141723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:02.142010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:02.142289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:02.176288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] MFA Rules not processed for user `aa0b5510413b4ee6b1dd0048d3df7c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:02.177543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:02.265793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1125/2218] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:02.277018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:02.278824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:02.348538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35e24-2ae4-433d-b87c-54325b1ebeb8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:02.352932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:02.353762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:02.354039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:02.354314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:02.425704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:02.427292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e17026-8bf7-4a99-a94b-fc38d3406540 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:02.439011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1094/2219] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:06.291988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:06.445146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78c294-a424-468b-92f6-88deac95b69a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:06.460761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:06.462768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:06.463659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:06.464350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] PATH_INFO: `/v3/users/e5d72758601b46359914918c442dc767` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:06.467671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d72758601b46359914918c442dc767)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:06.474789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:06.601117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:06.634449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d72758601b46359914918c442dc767'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:06.649129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d72758601b46359914918c442dc767'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:06.650092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d72758601b46359914918c442dc767'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:06.650983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d72758601b46359914918c442dc767'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:06.679761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d72758601b46359914918c442dc767'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:06.703542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1126/2220] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:06 2022] DELETE /identity/v3/users/e5d72758601b46359914918c442dc767 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:06.708798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:06.709631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:06.709631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:06.738091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:06.739393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:06.784854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:06.887005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1095/2221] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:06.922761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:06.925330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:07.047307 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da5aae-2c7c-4c62-b1d6-af29be1c8284 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:07.056912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:07.059009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:07.059895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:07.060420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:07.097153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:07.099783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:07.149532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1127/2222] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:07.551689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:07.553926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:07.571396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddcb5-6d0e-4d1f-b9f6-06e34700480c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:07.576596 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:07.577285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:07.577597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:07.577871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] PATH_INFO: `/v3/projects/ec22326a930d48b99f08f4826efc5f4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:07.579633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec22326a930d48b99f08f4826efc5f4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:07.581711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:07.675792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec22326a930d48b99f08f4826efc5f4a', 'request_id': 'req-941b7762-5f8f-4dc1-8519-db8e92bfd763'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.685645 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec22326a930d48b99f08f4826efc5f4a', 'request_id': 'req-941b7762-5f8f-4dc1-8519-db8e92bfd763'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.695197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec22326a930d48b99f08f4826efc5f4a', 'request_id': 'req-941b7762-5f8f-4dc1-8519-db8e92bfd763'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.714170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] The token cache is being invalidate because project ec22326a930d48b99f08f4826efc5f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:07.714978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.715629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.718387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1096/2223] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:07 2022] DELETE /identity/v3/projects/ec22326a930d48b99f08f4826efc5f4a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:07.724309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:07.738423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f8d7e-8d3e-4083-a1dd-a74677196853 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:07.829378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:07.830531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:07.831007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:07.831459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] PATH_INFO: `/v3/users/aa0b5510413b4ee6b1dd0048d3df7c06` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:07.834007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0b5510413b4ee6b1dd0048d3df7c06)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:07.838031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:07.946207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0b5510413b4ee6b1dd0048d3df7c06'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.963129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0b5510413b4ee6b1dd0048d3df7c06'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.964571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0b5510413b4ee6b1dd0048d3df7c06'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.966107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0b5510413b4ee6b1dd0048d3df7c06'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:07.978712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:07.988378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9551aa29-4ce9-47f2-9bd7-4da97a4d462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0b5510413b4ee6b1dd0048d3df7c06'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.038871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1128/2224] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:07 2022] DELETE /identity/v3/users/aa0b5510413b4ee6b1dd0048d3df7c06 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:45:08.504096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:08.506374 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:08.528685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b7762-5f8f-4dc1-8519-db8e92bfd763 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:08.607062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:08.608138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:08.608628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:08.609110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] PATH_INFO: `/v3/projects/1a19aa17d0b14412a4d696f14ea643b6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:08.611136 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a19aa17d0b14412a4d696f14ea643b6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:08.613152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:08.694992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:08.701672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a19aa17d0b14412a4d696f14ea643b6', 'request_id': 'req-a6ff934e-0588-4864-b431-210e0dd8337d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.718802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a19aa17d0b14412a4d696f14ea643b6', 'request_id': 'req-a6ff934e-0588-4864-b431-210e0dd8337d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.735383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a19aa17d0b14412a4d696f14ea643b6', 'request_id': 'req-a6ff934e-0588-4864-b431-210e0dd8337d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.752792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] The token cache is being invalidate because project 1a19aa17d0b14412a4d696f14ea643b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:08.753184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.754043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6ff934e-0588-4864-b431-210e0dd8337d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:08.762173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1097/2225] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:08 2022] DELETE /identity/v3/projects/1a19aa17d0b14412a4d696f14ea643b6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:08.765029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:08.766373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:08.766373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:08.796559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:08.797853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:08.896493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1129/2226] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:08.908481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:08.908738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:08.909037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:08.932505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:08.933779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:08.942825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1098/2227] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:08.947555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:08.958488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10484d-23bb-40ad-8d2f-ad13dfdb191e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:08.967358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:08.968360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:08.968814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:08.969261 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:08.970546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:08.971045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:08.973957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:08.993638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1130/2228] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:08.998314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.008641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144564a-2416-4399-b0f8-794bf707c03f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.013950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.014593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.014869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.015130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.015814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.017858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.049419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1099/2229] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:08 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:09.055636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.066560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03347-ee5d-4a8b-bd16-8f8021c65169 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.130979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.131611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.131892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.132166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.132879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.135288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.158957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1131/2230] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:09 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:09.163121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.175868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8bc37-4861-4155-9766-3beeb8013d4b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.181411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.182084 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.182363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.184127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.184127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.185069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.197630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1100/2231] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:09.201591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.213333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f254-ae5b-41f1-ac98-9afbc60e701f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.233242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.234028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.234196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.234472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.235195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.236828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33f621-4003-446b-9ddc-db6a33d4c59c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.256600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1132/2232] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:09.260607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.272603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2db2e-b103-404d-af84-2915293a828b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.283601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.284317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.284601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.284879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] PATH_INFO: `/v3/projects/0f8c987979314173a1de841f6406f391/users/ca356d7c1b7f4768adaa0c09daa3487b/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.302762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8c987979314173a1de841f6406f391, user_id=ca356d7c1b7f4768adaa0c09daa3487b, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.305466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.325666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1101/2233] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:09 2022] PUT /identity/v3/projects/0f8c987979314173a1de841f6406f391/users/ca356d7c1b7f4768adaa0c09daa3487b/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:09.334568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.336161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.336553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.374470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] MFA Rules not processed for user `ca356d7c1b7f4768adaa0c09daa3487b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:09.375685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:09.409232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:09.442839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1133/2234] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:09.455219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:09.457219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:09.524469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8c3b-4c7e-453e-bffb-a2b363fcfcdd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:09.530825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:09.531467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:09.532546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:09.532546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:09.541721 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:09.543184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7c78b-f157-41f3-920c-146b19a9807f service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:09.553434 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1102/2235] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:17.359434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:17.362765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:17.477338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7b004-caf2-4da7-844b-156d9d96ac10 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:17.490888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:17.491583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:17.491864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:17.492155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] PATH_INFO: `/v3/users/b544405861c64c1fbab8707e3b1e2151` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:17.493788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b544405861c64c1fbab8707e3b1e2151)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:17.496036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:17.580545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b544405861c64c1fbab8707e3b1e2151'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:17.600289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b544405861c64c1fbab8707e3b1e2151'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:17.602099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b544405861c64c1fbab8707e3b1e2151'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:17.603502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b544405861c64c1fbab8707e3b1e2151'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:17.622983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b544405861c64c1fbab8707e3b1e2151'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:17.652236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1134/2236] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:17 2022] DELETE /identity/v3/users/b544405861c64c1fbab8707e3b1e2151 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:17.659437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:17.660198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:17.660198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:17.696905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:17.700876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:17.812380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1103/2237] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:17.824309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:17.826166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:17.922309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333ceb-8238-4a9b-966a-e7d66af7308d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:17.929089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:17.930319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:17.930796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:17.931317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:17.943139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:17.944602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a8638-9b90-4367-b516-f1c1e7b86f80 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:17.949253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:17.963798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1135/2238] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:45:18.934584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:18.936677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:18.962999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4accb-8fed-4b43-adc3-b91e6ad894a9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:18.972053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:18.973118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:18.973626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:18.974106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] PATH_INFO: `/v3/projects/5bc61ee6b1ad4ad1a6e64810a6e282d5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:18.975733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc61ee6b1ad4ad1a6e64810a6e282d5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:18.978700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:18.990918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:19.087569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc61ee6b1ad4ad1a6e64810a6e282d5', 'request_id': 'req-4e132362-5aa0-4745-9673-54e4171825c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:19.102048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc61ee6b1ad4ad1a6e64810a6e282d5', 'request_id': 'req-4e132362-5aa0-4745-9673-54e4171825c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:19.112682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc61ee6b1ad4ad1a6e64810a6e282d5', 'request_id': 'req-4e132362-5aa0-4745-9673-54e4171825c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:19.133132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] The token cache is being invalidate because project 5bc61ee6b1ad4ad1a6e64810a6e282d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:19.133672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:19.134718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4e132362-5aa0-4745-9673-54e4171825c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:19.137226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1104/2239] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:18 2022] DELETE /identity/v3/projects/5bc61ee6b1ad4ad1a6e64810a6e282d5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:45:19.143074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.143533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.143768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.169778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:19.172276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:19.277937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1136/2240] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:19.287283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.287585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.287819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.318883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:19.320770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:19.331259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1105/2241] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:19.338382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.355823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887c256-71a6-4224-9d10-249ee0364146 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.363080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.363721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.375719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.375719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.375719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:19.375719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.375719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.385622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1137/2242] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:19.389264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.405336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7c8cb-c0e8-4b6b-954d-2178c3c5ff74 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.412676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.413492 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.413872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.414306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.415034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.417218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.467091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1106/2243] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:19 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:45:19.472982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.484380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec7aa2-22c3-4e7e-a572-c6c80a26cc8c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.583503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.584358 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.584783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.585061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.586243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.590836 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.648953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1138/2244] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:19 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:19.649742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.678411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb626-2235-47be-b2f1-46253fcbbb1f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.686632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.688878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.690173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.690173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.691520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.694425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.719875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1107/2245] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:19.721236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.740881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6854066-4412-4e7b-a3a8-5f0b46cde3e8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.746498 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.747346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.747717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.748079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.748881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.750894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d657-5f1d-4c91-b6de-5cff8b3c65c4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.763707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1139/2246] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:45:19.768616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:19.786709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39846b5-d02f-468d-a50f-8bbe544ae8f5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:19.794901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:19.796063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.796507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.796933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] PATH_INFO: `/v3/projects/1d2d7b5d89da449da7df7657fecc0160/users/c70079d1e0ba4c249c9b46eba2bae6e7/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.823834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2d7b5d89da449da7df7657fecc0160, user_id=c70079d1e0ba4c249c9b46eba2bae6e7, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:19.828903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:19.857742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1108/2247] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:19 2022] PUT /identity/v3/projects/1d2d7b5d89da449da7df7657fecc0160/users/c70079d1e0ba4c249c9b46eba2bae6e7/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:19.874271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1ee7a-73d2-4096-b860-d4365cd3a094 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:19.875187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1ee7a-73d2-4096-b860-d4365cd3a094 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:19.875776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1ee7a-73d2-4096-b860-d4365cd3a094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:19.960637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8be1ee7a-73d2-4096-b860-d4365cd3a094 None None] MFA Rules not processed for user `c70079d1e0ba4c249c9b46eba2bae6e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:19.962830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8be1ee7a-73d2-4096-b860-d4365cd3a094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:20.083975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:20.110355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1140/2248] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:20.128885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:20.132044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:20.268438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe722-9971-41cf-ad5d-ba0c9d316c49 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:20.279470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:20.282460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:20.282460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:20.282800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:20.311798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:20.313293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf33d0-3448-4195-ae90-83b3827e3db5 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:20.329715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1109/2249] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:21.445005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:21.445619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:21.446095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:21.474073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] MFA Rules not processed for user `26c501f3361e46239a6a492d0858afe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:21.475811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:21.570895 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1141/2250] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:30.320867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:31.559286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=28 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:39.642797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ce2a-714b-41f5-90be-a0424c58e1ca None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:39.643648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ce2a-714b-41f5-90be-a0424c58e1ca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:39.643847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ce2a-714b-41f5-90be-a0424c58e1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:39.686005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3662ce2a-714b-41f5-90be-a0424c58e1ca None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:39.687441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3662ce2a-714b-41f5-90be-a0424c58e1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:39.817763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1110/2251] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:39.913428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:39.916512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:40.051968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c6970-97a5-4935-98ca-d606f5a3843d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:40.062281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:40.063348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:40.063694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:40.064092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:40.077483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:40.080413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:40.090321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1142/2252] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:45:41.158042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ae4ac-d162-4de1-b976-ce353bb24f2e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:41.158674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ae4ac-d162-4de1-b976-ce353bb24f2e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:41.158954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ae4ac-d162-4de1-b976-ce353bb24f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:41.193514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a42ae4ac-d162-4de1-b976-ce353bb24f2e None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:41.195714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a42ae4ac-d162-4de1-b976-ce353bb24f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:41.209110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1111/2253] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:45:41.232252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:41.253151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650cf1-1ffd-46cf-b9ea-d5282eb16be0 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:41.260041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:41.260817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:41.261606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:41.261969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:41.275184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:41.277072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:41.289729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1143/2254] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:45:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:45:41.631572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95db7-cbe5-407c-867f-28ec293d1195 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:41.632624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95db7-cbe5-407c-867f-28ec293d1195 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:41.633169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95db7-cbe5-407c-867f-28ec293d1195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:41.666942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fdf95db7-cbe5-407c-867f-28ec293d1195 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:41.669462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fdf95db7-cbe5-407c-867f-28ec293d1195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:41.681655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1112/2255] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:41.695594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:41.708805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb9a0-1535-4a50-a4a1-cb85e8cb5c33 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:41.714533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:41.715178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:41.715459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:41.716094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:41.729367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:41.732248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:41.746862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1144/2256] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:45:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:45:42.290597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:42.290597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:42.290597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:42.313326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:42.314947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:42.331934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1113/2257] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:42.342099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:42.360314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e40ff0-6f37-4fe5-9b08-99a2e2e947ca service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:42.368393 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:42.369709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:42.370204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:42.370805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:42.386208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:42.389128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf97509-6abd-472f-902e-abcd8affcecf service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:42.404010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1145/2258] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:45:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:45:43.421303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:43.552936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7923ca-0eb3-4fe1-9ecf-21f46bbf71aa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:43.565126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:43.566150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:43.566551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:43.567100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] PATH_INFO: `/v3/users/c70079d1e0ba4c249c9b46eba2bae6e7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:43.568762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c70079d1e0ba4c249c9b46eba2bae6e7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:43.572219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:43.691955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c70079d1e0ba4c249c9b46eba2bae6e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:43.693379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c70079d1e0ba4c249c9b46eba2bae6e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:43.719018 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c70079d1e0ba4c249c9b46eba2bae6e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:43.733680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c70079d1e0ba4c249c9b46eba2bae6e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:43.735110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c70079d1e0ba4c249c9b46eba2bae6e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:43.768706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1114/2259] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:43 2022] DELETE /identity/v3/users/c70079d1e0ba4c249c9b46eba2bae6e7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:43.770188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:43.771472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:43.771472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:43.816411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:43.817633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:44.040653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1146/2260] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:44.078495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:44.081725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:44.115519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:44.280577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee563-1f55-40f0-aa2f-5240c05534e1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:44.309670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:44.310845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:44.311381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:44.313179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:44.336044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:44.339095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c82a7-8369-4654-a79b-3cda742f3f4b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:44.372338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1115/2261] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:44.916804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:44.918893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:44.939340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6543136-48b6-4d1a-96ea-0b69b33d6499 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:44.947130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:44.948181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:44.948531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:44.948828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] PATH_INFO: `/v3/projects/1d2d7b5d89da449da7df7657fecc0160` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:44.950310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2d7b5d89da449da7df7657fecc0160)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:44.952628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.004384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:45.009454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2d7b5d89da449da7df7657fecc0160', 'request_id': 'req-0b32fddd-1220-4093-bb94-dde7dff4035e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.028116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2d7b5d89da449da7df7657fecc0160', 'request_id': 'req-0b32fddd-1220-4093-bb94-dde7dff4035e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.052654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2d7b5d89da449da7df7657fecc0160', 'request_id': 'req-0b32fddd-1220-4093-bb94-dde7dff4035e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.070883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] The token cache is being invalidate because project 1d2d7b5d89da449da7df7657fecc0160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:45.071166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.071887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b32fddd-1220-4093-bb94-dde7dff4035e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.073457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1147/2262] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:44 2022] DELETE /identity/v3/projects/1d2d7b5d89da449da7df7657fecc0160 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:45.080817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.081382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.081653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.117924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:45.117924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:45.192989 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1116/2263] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:45:45.203230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.203755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.204020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.229531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:45.230821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:45.240421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1148/2264] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:45.245261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.256149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593e430-f76a-4378-b4e1-b8a6183dd3f9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.261243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.261948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.262229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.262510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.263235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:45.263539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.265182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.292147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1117/2265] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:45.293683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.307650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5e08-6d86-421c-b6f8-3e6ddb5f9cb3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.312500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.313578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.314077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.314556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.315766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.318491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.359202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1149/2266] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:45 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:45.365452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.378970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568396-8873-4c39-9c57-8f972769521c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.478277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.479011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.479364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.479724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.483401 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.487970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.527548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1118/2267] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:45 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:45.528747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.548595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7732c-863a-45bf-adc1-9a9cc4669f22 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.566378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.569045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.569869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.571733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.574624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.580184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.597501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1150/2268] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:45.609311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.632133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c1d2-6d6f-4a3e-bdcb-e429e0cef63e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.644585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.645804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.646156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.646513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.647409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.649689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.664425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1119/2269] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:45.669323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.696402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1490bb-3c66-4642-8c6f-755ebae43053 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.705381 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.706563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.707081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.707604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] PATH_INFO: `/v3/projects/dcf006ee932d483b9cb06f83b7b86b47/users/bdfea4c2f4f94cec9cabde2c7822b9a9/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.744076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:45.747127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:45.757074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf006ee932d483b9cb06f83b7b86b47, user_id=bdfea4c2f4f94cec9cabde2c7822b9a9, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.766525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d19b9-01c3-4efc-a68e-435c41ef65e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.777299 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b80b7e-12b2-4ed8-94cf-67c88526b8b0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:45.791533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:45.792691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.793871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.794384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] PATH_INFO: `/v3/users/c3c864261ae94460ad72e2c3b709d8e3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.797810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c864261ae94460ad72e2c3b709d8e3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:45.800782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:45.814920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1151/2270] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:45 2022] PUT /identity/v3/projects/dcf006ee932d483b9cb06f83b7b86b47/users/bdfea4c2f4f94cec9cabde2c7822b9a9/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:45:45.820323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:45.821549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:45.822134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:45.828548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:45.883022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:45.887120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:45.956915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c864261ae94460ad72e2c3b709d8e3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.960677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c864261ae94460ad72e2c3b709d8e3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.978115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c864261ae94460ad72e2c3b709d8e3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.994601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c864261ae94460ad72e2c3b709d8e3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:45.995919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e34c3f06-43b8-4ef5-9c48-46849c9dbaaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c864261ae94460ad72e2c3b709d8e3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:46.042999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1120/2271] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:45 2022] DELETE /identity/v3/users/c3c864261ae94460ad72e2c3b709d8e3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:45:46.048105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1152/2272] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:46.050546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:46.051045 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:46.051652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:46.092219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:46.093931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:46.111921 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:46.114016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:46.334610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1121/2273] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:46.347368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:46.349698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:46.398441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bad7c5-6084-437e-aa2b-3d77f96e39ab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:46.467515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:46.529321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:46.529997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:46.530458 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:46.530743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:46.559578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:46.561129 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:46.589802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1153/2274] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:45:46.602090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90af76-2ad5-47e9-bdc7-57768d2896f1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:46.611830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:46.613760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:46.614464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:46.615264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:46.636154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:46.640734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2837b-0008-47cc-a50f-ceef842f4931 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:46.661279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1122/2275] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:45:46.911859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:46.914121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:46.935994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007d24-9b25-4516-861a-dc7e5cdea0cb service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:46.943984 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:46.944884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:46.945124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:46.945502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] PATH_INFO: `/v3/projects/9e89f6a32366452592eb5a76f6af57fe` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:46.947172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e89f6a32366452592eb5a76f6af57fe)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:46.950107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:47.014578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e89f6a32366452592eb5a76f6af57fe', 'request_id': 'req-bde2c072-c89f-4052-92b2-f0c6557f3376'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:47.028421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e89f6a32366452592eb5a76f6af57fe', 'request_id': 'req-bde2c072-c89f-4052-92b2-f0c6557f3376'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:47.035591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e89f6a32366452592eb5a76f6af57fe', 'request_id': 'req-bde2c072-c89f-4052-92b2-f0c6557f3376'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:47.049627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] The token cache is being invalidate because project 9e89f6a32366452592eb5a76f6af57fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:47.051522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:47.052343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:47.054493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1154/2276] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:46 2022] DELETE /identity/v3/projects/9e89f6a32366452592eb5a76f6af57fe => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:52.894617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:52.895109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:52.895205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:52.947768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] MFA Rules not processed for user `bdfea4c2f4f94cec9cabde2c7822b9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:52.948435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:52.976316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:53.097389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1123/2277] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:53.102424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.104582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:53.198285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c072-c89f-4052-92b2-f0c6557f3376 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.205429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.206409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.206722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.206994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.207834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.209797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.256931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1155/2278] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:53 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:53.261160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.303888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e591e-b4b4-414c-98b8-6e038dcaa754 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.433487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.434318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.434771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.435139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.435931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.438028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.471619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1124/2279] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:53 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:53.476064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.488153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a79b8-7441-4a53-9f52-df6fc713559b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.494779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.495919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.496494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.496814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.497828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.500919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.517613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1156/2280] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:53.520816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.536660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bff7-ed41-4e16-87b6-898489be4494 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.541371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.542202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.542757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.543896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] PATH_INFO: `/v3/projects/0ee7590f51f948e1b165243adea8979e/users/f22cf65576c6410697a0313785b3a621/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.562824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee7590f51f948e1b165243adea8979e, user_id=f22cf65576c6410697a0313785b3a621, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.565631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.582630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1125/2281] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:53 2022] PUT /identity/v3/projects/0ee7590f51f948e1b165243adea8979e/users/f22cf65576c6410697a0313785b3a621/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:53.587830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.599171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5e31-1413-40eb-a17c-a48144f6f289 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.630879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:53.682877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.683798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.684289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.684758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.685947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.688713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.707274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1157/2282] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:45:53.711767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:53.723913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea8eb2-c279-43ec-b8c9-68b4019dd2d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:53.729709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:53.730430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:53.730706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:53.730973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] PATH_INFO: `/v3/projects/0ee7590f51f948e1b165243adea8979e/users/f22cf65576c6410697a0313785b3a621/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:53.733101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee7590f51f948e1b165243adea8979e, user_id=f22cf65576c6410697a0313785b3a621, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:53.735832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644cff3-5721-4ddb-9d59-61a83be2fe49 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:53.749326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1126/2283] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:53 2022] PUT /identity/v3/projects/0ee7590f51f948e1b165243adea8979e/users/f22cf65576c6410697a0313785b3a621/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:54.012396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:54.015748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:54.043042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e8479-0dfc-4777-a349-65534a201e12 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:54.124256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:54.124256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:54.124860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:54.125052 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] PATH_INFO: `/v3/users/d472a2b220f0412ba5c2b0f8b06fe839` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:54.126351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d472a2b220f0412ba5c2b0f8b06fe839)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:54.128399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:54.204733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd472a2b220f0412ba5c2b0f8b06fe839'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:54.219292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd472a2b220f0412ba5c2b0f8b06fe839'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:54.220051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd472a2b220f0412ba5c2b0f8b06fe839'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:54.220667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd472a2b220f0412ba5c2b0f8b06fe839'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:54.261168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd472a2b220f0412ba5c2b0f8b06fe839'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:54.287330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1158/2284] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:54 2022] DELETE /identity/v3/users/d472a2b220f0412ba5c2b0f8b06fe839 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:54.288878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610403f-0191-452e-9981-1a45921b8111 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:54.289724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610403f-0191-452e-9981-1a45921b8111 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:54.290220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610403f-0191-452e-9981-1a45921b8111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:54.335147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4610403f-0191-452e-9981-1a45921b8111 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:54.336753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4610403f-0191-452e-9981-1a45921b8111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:54.461621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1127/2285] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:45:54.469727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:54.470916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:54.642216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:54.710830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce347a-d9f0-4128-a24b-d516cf4e3fbd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:54.728336 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:54.729305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:54.729933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:54.730407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:54.756125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:54.760098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a977bc-80a8-42ac-a0f4-ff89abfc2b25 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:54.797830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1159/2286] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:45:55.509250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610403f-0191-452e-9981-1a45921b8111 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:55.511438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4610403f-0191-452e-9981-1a45921b8111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:55.527158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610403f-0191-452e-9981-1a45921b8111 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:55.533140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:55.534154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:55.534634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:55.535103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] PATH_INFO: `/v3/projects/bd252879187e4352902e9d8044463d92` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:55.550344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd252879187e4352902e9d8044463d92)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:55.555047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:55.570481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:55.653307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd252879187e4352902e9d8044463d92', 'request_id': 'req-ad55fd73-7a15-4d92-83ff-be5027995f7a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:55.663380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd252879187e4352902e9d8044463d92', 'request_id': 'req-ad55fd73-7a15-4d92-83ff-be5027995f7a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:55.671712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd252879187e4352902e9d8044463d92', 'request_id': 'req-ad55fd73-7a15-4d92-83ff-be5027995f7a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:55.684779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] The token cache is being invalidate because project bd252879187e4352902e9d8044463d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:55.685433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:55.686364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ad55fd73-7a15-4d92-83ff-be5027995f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:55.688635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1128/2287] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:55 2022] DELETE /identity/v3/projects/bd252879187e4352902e9d8044463d92 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:55.695463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:55.695959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:55.696182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:55.717491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:55.719310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:55.827076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1160/2288] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:55.834799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:55.835450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:55.835617 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:55.876058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:55.877243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:55.887722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1129/2289] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:55.892293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:55.907002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58669466-d8a5-4791-be7a-9c799459f26a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:55.917951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:55.918786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:55.919157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:55.920211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:55.922770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:55.923246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:55.927252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:55.963312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1161/2290] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:55.967927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:55.992481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf1612-5d33-4dd0-932b-701b8cfd1b90 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.008480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.010315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.011019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.011888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.013089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.015622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:56.142929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1130/2291] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:55 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:56.147878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:56.166854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef17d-4f5d-4b02-b9c3-fb7c559f5893 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.335754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.335754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.335754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.336121 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.336775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.353722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:56.406690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1162/2292] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:56 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:56.413730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:56.441053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba500-e656-4096-b8c5-742937dd3ea2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.450303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.452310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.452937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.453522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.455003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.458612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:56.485511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1131/2293] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:56.489948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:56.503465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56cdd-20f0-4c01-88d4-60ab27b0c441 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.511651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.513101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.513663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.514225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.515969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.519496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1046a9-7749-4f88-9e8b-299944fe81fb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:56.536605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1163/2294] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:45:56.540841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:56.555870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da6294-6883-452d-b7fd-c88427f55a16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.562428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.563752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.564091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.564517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] PATH_INFO: `/v3/projects/e65eb06a29924c39869f0e9cf3592ad5/users/2e40632351b94eaf9c29a537bd97084a/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.591431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65eb06a29924c39869f0e9cf3592ad5, user_id=2e40632351b94eaf9c29a537bd97084a, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.596701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:56.622126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1132/2295] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:45:56 2022] PUT /identity/v3/projects/e65eb06a29924c39869f0e9cf3592ad5/users/2e40632351b94eaf9c29a537bd97084a/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:56.624340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.624629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.624902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.685839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] MFA Rules not processed for user `2e40632351b94eaf9c29a537bd97084a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:56.687275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:56.782124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:56.818437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1164/2296] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:56.841209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:56.844413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:56.949354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a414e8a-5e51-4630-9033-b000dddfb8e9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:56.959929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:56.965318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:56.966433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:56.966960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:56.992347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:56.994747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535702f-8a07-4ca6-b75f-f27957b7ab62 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:57.007414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1133/2297] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:57.159793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:57.161407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:57.313644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3192c-ae2d-43ae-9de5-0b1d47e462cb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:57.326638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:57.328077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:57.329197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:57.329802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] PATH_INFO: `/v3/users/ca356d7c1b7f4768adaa0c09daa3487b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:57.334842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca356d7c1b7f4768adaa0c09daa3487b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:57.342228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:57.539055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca356d7c1b7f4768adaa0c09daa3487b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:57.566259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca356d7c1b7f4768adaa0c09daa3487b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:57.567527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca356d7c1b7f4768adaa0c09daa3487b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:57.574847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca356d7c1b7f4768adaa0c09daa3487b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:57.626518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca356d7c1b7f4768adaa0c09daa3487b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:57.655649 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1165/2298] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:45:57 2022] DELETE /identity/v3/users/ca356d7c1b7f4768adaa0c09daa3487b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:57.658359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:57.658870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:57.659260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:57.749289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:57.764619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:57.879266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:57.967405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1134/2299] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:57.992492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:57.995780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:58.152145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b548eab-fb93-4203-8517-c34d4404f756 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:58.159795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:58.161696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:58.162417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:58.163172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:58.183499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:58.187097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03293eac-5660-4087-be52-116f03ca2ee2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:58.218468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1166/2300] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:45:58.995872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:58.998193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:59.027019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199c309-6782-4edb-b97e-d18c36f58919 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:59.032689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:59.033768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.034332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.034999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] PATH_INFO: `/v3/projects/0f8c987979314173a1de841f6406f391` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.036538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8c987979314173a1de841f6406f391)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:59.038762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:59.113043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8c987979314173a1de841f6406f391', 'request_id': 'req-6e0534d2-0742-4881-8dc7-60f79c32729b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:59.124043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8c987979314173a1de841f6406f391', 'request_id': 'req-6e0534d2-0742-4881-8dc7-60f79c32729b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:59.141436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8c987979314173a1de841f6406f391', 'request_id': 'req-6e0534d2-0742-4881-8dc7-60f79c32729b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:59.159912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] The token cache is being invalidate because project 0f8c987979314173a1de841f6406f391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:45:59.160194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:59.160897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6e0534d2-0742-4881-8dc7-60f79c32729b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:45:59.162970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1135/2301] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:45:58 2022] DELETE /identity/v3/projects/0f8c987979314173a1de841f6406f391 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:45:59.168619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.170839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.171629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.196027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:45:59.244988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:59.255105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:59.484042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1167/2302] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:59.493119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8600-af75-459d-84e8-829e63349b47 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.494008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8600-af75-459d-84e8-829e63349b47 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.494915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8600-af75-459d-84e8-829e63349b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.563095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-465c8600-af75-459d-84e8-829e63349b47 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:45:59.565879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-465c8600-af75-459d-84e8-829e63349b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:45:59.592882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1136/2303] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:45:59.598041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:59.616422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c8e8b-b65a-432a-8580-f8bb8f30f9d7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:59.624870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:59.625169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.625513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.625847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.637869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:45:59.638423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:59.651735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:59.679658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1168/2304] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:45:59.685286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8600-af75-459d-84e8-829e63349b47 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:59.707308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8600-af75-459d-84e8-829e63349b47 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:59.719868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:59.721079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.721738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.722277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.724082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:59.728028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:45:59.809871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1137/2305] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:45:59 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:45:59.810203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:45:59.825926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59c8c4-8bc9-4b18-bcf4-b43b34a75bb7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:45:59.970876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:45:59.972201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:45:59.972484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:45:59.972758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:45:59.977920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:45:59.980021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.033448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1169/2306] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:45:59 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:46:00.034515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:00.049118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3115-b912-416d-9054-ba1da2f43636 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:00.070926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:00.071739 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.072065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.072409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.073207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:00.075296 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.099639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1138/2307] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:46:00.105494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:00.127883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd157d4-6f06-45a1-ae69-77af6e595760 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:00.145867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:00.146800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.147264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.147702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.148605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:00.150640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2a0c5-babc-4061-9e0b-f1b8e2f8da66 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.192269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1170/2308] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:46:00.194593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:00.212849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3757b-138c-4c08-a337-392363360e7c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:00.227867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:00.228913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.228913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.229216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] PATH_INFO: `/v3/projects/2473d67e62bb49b98f14f2ab3a69ea55/users/85931e4ae1ca40739f9fa1dc65bbba24/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.264626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2473d67e62bb49b98f14f2ab3a69ea55, user_id=85931e4ae1ca40739f9fa1dc65bbba24, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:00.275478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.302613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1139/2309] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:46:00 2022] PUT /identity/v3/projects/2473d67e62bb49b98f14f2ab3a69ea55/users/85931e4ae1ca40739f9fa1dc65bbba24/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:46:00.315881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.315881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.315881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.386204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] MFA Rules not processed for user `85931e4ae1ca40739f9fa1dc65bbba24`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:00.387736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:00.475654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:00.491608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1171/2310] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:46:00.499855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:00.502895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:00.607367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50332-0635-4bef-8e9a-4ed07605b71a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:00.613118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:00.614089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.614574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.615050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.626133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:00.631688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:00.634396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581bea8-1bb9-4452-86bc-a2eb8aec7c52 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.647431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1140/2311] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:00.680969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:00.791788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25a4a7-b0c0-4e7e-b3e3-6a87de36961e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:00.796622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:00.799271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:00.799662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:00.800096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] PATH_INFO: `/v3/users/85931e4ae1ca40739f9fa1dc65bbba24` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:00.803612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85931e4ae1ca40739f9fa1dc65bbba24)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:00.807085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:00.937524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85931e4ae1ca40739f9fa1dc65bbba24'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:00.955269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85931e4ae1ca40739f9fa1dc65bbba24'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:00.956345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85931e4ae1ca40739f9fa1dc65bbba24'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:00.958465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85931e4ae1ca40739f9fa1dc65bbba24'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:00.994487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85931e4ae1ca40739f9fa1dc65bbba24'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:01.093125 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1172/2312] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:46:00 2022] DELETE /identity/v3/users/85931e4ae1ca40739f9fa1dc65bbba24 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:46:01.108305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c7fc-4ac6-48b9-875f-b06549cdb2ed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:01.108789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c7fc-4ac6-48b9-875f-b06549cdb2ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:01.108904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c7fc-4ac6-48b9-875f-b06549cdb2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:01.129795 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6cd7c7fc-4ac6-48b9-875f-b06549cdb2ed None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:01.131254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6cd7c7fc-4ac6-48b9-875f-b06549cdb2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:01.340076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1141/2313] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:01.394356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:01.396957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:01.554484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce36520-b0cd-46a9-8d52-21e8b8c85ed8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:01.560490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:01.561681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:01.562343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:01.562979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:01.593240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:01.595531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:01.631490 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1173/2314] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:02.574235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:02.576206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:02.577195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:02.676079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] MFA Rules not processed for user `f22cf65576c6410697a0313785b3a621`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:02.677649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:02.789329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1142/2315] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:02.830412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:02.832804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:02.884877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:02.995194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650cdd-8442-45b8-b1d2-342409a883cf service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:02.995946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:02.997367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:03.009388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:03.010233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:03.010893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:03.011178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:03.014785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:03.020023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf322-6a42-4af4-91d8-1c2e6a370847 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:03.026935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:03.028081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:03.028672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:03.029112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] PATH_INFO: `/v3/projects/2473d67e62bb49b98f14f2ab3a69ea55` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:03.030828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2473d67e62bb49b98f14f2ab3a69ea55)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:03.033891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:03.110409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:03.112216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dc1b5-aef6-4ff2-8618-c476e49e5517 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:03.130548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1174/2316] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:03.144825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2473d67e62bb49b98f14f2ab3a69ea55', 'request_id': 'req-66a9a3ae-ff55-4289-9dab-81c6beb62b51'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:03.165776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2473d67e62bb49b98f14f2ab3a69ea55', 'request_id': 'req-66a9a3ae-ff55-4289-9dab-81c6beb62b51'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:03.174920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2473d67e62bb49b98f14f2ab3a69ea55', 'request_id': 'req-66a9a3ae-ff55-4289-9dab-81c6beb62b51'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:03.202324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] The token cache is being invalidate because project 2473d67e62bb49b98f14f2ab3a69ea55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:46:03.202641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:03.203336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:03.206002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1143/2317] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:46:02 2022] DELETE /identity/v3/projects/2473d67e62bb49b98f14f2ab3a69ea55 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:46:05.412718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf09fb-acd2-44ff-8e5b-8802469bdc9b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:05.413883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf09fb-acd2-44ff-8e5b-8802469bdc9b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:05.413883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf09fb-acd2-44ff-8e5b-8802469bdc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:05.455203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cadf09fb-acd2-44ff-8e5b-8802469bdc9b None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:05.457911 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cadf09fb-acd2-44ff-8e5b-8802469bdc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:05.606793 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1175/2318] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:46:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:05.624784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:05.628549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:05.640397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:05.799801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34363-6f10-4e0f-830a-030a99f8154d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:05.801148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34363-6f10-4e0f-830a-030a99f8154d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:05.801148 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb34363-6f10-4e0f-830a-030a99f8154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:05.881440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4bb34363-6f10-4e0f-830a-030a99f8154d None None] MFA Rules not processed for user `bdfea4c2f4f94cec9cabde2c7822b9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:05.883455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4bb34363-6f10-4e0f-830a-030a99f8154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:05.905833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a3ae-ff55-4289-9dab-81c6beb62b51 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:05.926464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:05.928984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:05.930109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:05.930895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:05.948726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:05.952379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:05.973829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1144/2319] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:46:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:46:06.056246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1176/2320] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:12.817515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:12.819702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:12.975317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a60bb-0849-453f-8183-1a20eda0f63c service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:12.989540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:12.990597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:12.990947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:12.991291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] PATH_INFO: `/v3/users/2e40632351b94eaf9c29a537bd97084a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:12.992689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e40632351b94eaf9c29a537bd97084a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:12.995005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:13.111856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e40632351b94eaf9c29a537bd97084a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:13.114831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e40632351b94eaf9c29a537bd97084a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:13.155097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e40632351b94eaf9c29a537bd97084a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:13.190301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e40632351b94eaf9c29a537bd97084a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:13.194598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e40632351b94eaf9c29a537bd97084a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:13.250356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1145/2321] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:46:12 2022] DELETE /identity/v3/users/2e40632351b94eaf9c29a537bd97084a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:46:13.252239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:13.253213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:13.253213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:13.287058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:13.311330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:13.313715 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:13.463050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1177/2322] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:13.477465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:13.479731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:13.522667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:13.524202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:13.526544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:13.663812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d1ad-d8d2-432a-9d03-2eb917c01501 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:13.675998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:13.677526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:13.678626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:13.679852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:13.702241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:13.704682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:13.728086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1146/2323] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:13.755018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54483-a916-471d-a49d-4e03ab3fd9b2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:13.788037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:13.798113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:13.798397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:13.798694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:13.809105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:13.851468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:13.854041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37424dd-1a14-42cd-a5b8-50e80d9bd143 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:13.893484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1178/2324] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:14.668653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:14.672013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:14.690108 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef541e55-0e20-43b8-99b1-c717fa4d975a service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:14.696658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:14.698060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:14.698519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:14.699290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] PATH_INFO: `/v3/projects/e65eb06a29924c39869f0e9cf3592ad5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:14.701502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65eb06a29924c39869f0e9cf3592ad5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:14.704469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:14.834681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65eb06a29924c39869f0e9cf3592ad5', 'request_id': 'req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:14.854150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65eb06a29924c39869f0e9cf3592ad5', 'request_id': 'req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:14.870431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65eb06a29924c39869f0e9cf3592ad5', 'request_id': 'req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:14.906931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] The token cache is being invalidate because project e65eb06a29924c39869f0e9cf3592ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:46:14.907357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:14.908137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7745cc-0fef-491f-9b8b-4ab64cb86ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:46:14.910610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1147/2325] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:46:14 2022] DELETE /identity/v3/projects/e65eb06a29924c39869f0e9cf3592ad5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:46:14.916284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:14.916925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:14.916925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:14.944039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:14.945406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:15.038219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1179/2326] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:46:15.046238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.046548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.046821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.070790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:15.072023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:15.088417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1148/2327] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:15.089522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.101651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929df467-1b6c-4c3d-9007-d9d29583b967 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.109155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.109968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.110350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.110721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.111559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:46:15.111969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.114138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.135708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1180/2328] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:46:15.140688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.154197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09349a-af0b-4535-98b3-cbdf3a082aca None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.168309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.169161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.173053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.173053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.173053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.174400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.227339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1149/2329] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:46:15 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:46:15.232871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.272999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2832-db4b-47d0-a62a-ec6f24a4eef3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.328405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:15.391538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.392152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.392430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.392713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.393892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.395933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.449306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1181/2330] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:46:15 2022] POST /identity/v3/users => generated 291 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:46:15.456072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.473278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17403430-c5ab-45ca-ac7f-b94c8debe77f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.484202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.485157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.486831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.488945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.490974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.495684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.510495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1150/2331] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:46:15.519608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.540653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3bf3a-c90a-491b-b334-28cbd0c7a28e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.556556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.557697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.558500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.558988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.560298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.563626 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2ddab-1c4f-4228-93bf-80a36471e44b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.585673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1182/2332] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:46:15.590748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.615719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eedf0b-240f-4092-b834-73ab789db247 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:15.630587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:15.631695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.632189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.632676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] PATH_INFO: `/v3/projects/3b04d30f1426408ba5c8fa0238982089/users/c362f3a63b944fbcbea080d55c8fdfa1/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.662215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b04d30f1426408ba5c8fa0238982089, user_id=c362f3a63b944fbcbea080d55c8fdfa1, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:15.666959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:15.704482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1151/2333] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:46:15 2022] PUT /identity/v3/projects/3b04d30f1426408ba5c8fa0238982089/users/c362f3a63b944fbcbea080d55c8fdfa1/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:46:15.710204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457753-b5ad-4b3a-a9e5-b61417bc4cd1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:15.710559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457753-b5ad-4b3a-a9e5-b61417bc4cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:15.710884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457753-b5ad-4b3a-a9e5-b61417bc4cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:15.749239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-61457753-b5ad-4b3a-a9e5-b61417bc4cd1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:15.750942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-61457753-b5ad-4b3a-a9e5-b61417bc4cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:15.883439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1183/2334] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:15.907604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:15.922372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:16.036901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:16.078787 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367855ab-6c20-4962-b280-b1f882af158a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:16.085529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:16.086991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:16.087690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:16.088245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:16.109945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:16.113475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c2e08-8160-42ec-8890-3bd82ea65118 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:16.135221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1152/2335] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:16.941426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:16.942277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:16.942277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:16.987095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] MFA Rules not processed for user `a8bbeb0e65a54b1199d0fe718168af70`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:16.988247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:17.104089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1184/2336] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:17.707693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814f23-dac0-496e-b2e8-08baf7651e88 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:17.708224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814f23-dac0-496e-b2e8-08baf7651e88 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:17.708327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814f23-dac0-496e-b2e8-08baf7651e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:17.772387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-15814f23-dac0-496e-b2e8-08baf7651e88 None None] MFA Rules not processed for user `c362f3a63b944fbcbea080d55c8fdfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:17.774215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-15814f23-dac0-496e-b2e8-08baf7651e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:17.933576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1153/2337] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:17.949778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:17.953156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:18.115402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:18.119712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c7c85-2963-44f2-8a55-7b083cc5e6d0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:18.125011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:18.126116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:18.132114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:18.132596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:18.146488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:18.148705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25b2ce-ed1e-4c5f-b079-85ecd7181aa8 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:18.159749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1185/2338] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:20.077618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:20.079620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:20.080251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:20.126902 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] MFA Rules not processed for user `c362f3a63b944fbcbea080d55c8fdfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:20.128496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:20.138629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1154/2339] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:46:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:26.996242 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf690f-c186-46b8-b994-4df335da1068 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:26.996939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf690f-c186-46b8-b994-4df335da1068 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:26.997228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf690f-c186-46b8-b994-4df335da1068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:27.052479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-11cf690f-c186-46b8-b994-4df335da1068 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:27.054653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11cf690f-c186-46b8-b994-4df335da1068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:27.188013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1186/2340] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:46:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:27.208260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:27.210781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:27.352193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3255257-3022-43a0-abdf-ecf7095154c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:27.358817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:27.359804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:27.360227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:27.360703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:27.372050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:27.373758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183470df-6e94-486f-a3ea-7fdcbc989b87 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:27.383790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1155/2341] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:46:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:46:34.091982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.092645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.092645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.094855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1187/2342] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:46:34.103743 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907fa45-2372-4c67-bc8d-3c093232308d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.104723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907fa45-2372-4c67-bc8d-3c093232308d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.105212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907fa45-2372-4c67-bc8d-3c093232308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.145779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6907fa45-2372-4c67-bc8d-3c093232308d None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:34.153311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6907fa45-2372-4c67-bc8d-3c093232308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:34.269460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:34.349573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1156/2343] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:46:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:46:34.350801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:34.352688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:34.465413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126ae90-dcea-41c4-8a8a-2615eebf3df6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:34.478292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:34.478973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.479311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.480361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.494849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:34.497725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb2e66-8753-44b8-9f70-71c6a8fdf478 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:34.526001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1188/2344] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:46:34.638819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.639987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.640717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.642718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1157/2345] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:46:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:46:34.656913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97ec1-db69-40a1-b402-d2aa3b03295c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.657402 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97ec1-db69-40a1-b402-d2aa3b03295c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.657917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97ec1-db69-40a1-b402-d2aa3b03295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.717207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-99a97ec1-db69-40a1-b402-d2aa3b03295c None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:34.720003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-99a97ec1-db69-40a1-b402-d2aa3b03295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:34.758301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1189/2346] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:46:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:34.782873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:34.786519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:34.822539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed162a-e052-49bc-b675-b6b9fecc6571 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:34.827719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:34.828557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:34.829000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:34.829331 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:34.840653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:34.842493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d83c38-34bb-4528-8773-33037475257a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:34.860473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1158/2347] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:35.098515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f41c6-6de4-4f7d-a640-a0267b8fd6be None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:35.099248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f41c6-6de4-4f7d-a640-a0267b8fd6be None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:35.099396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f41c6-6de4-4f7d-a640-a0267b8fd6be None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:35.100888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1190/2348] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:46:35.119488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5c13-b8d1-4627-b758-d46f85c8ead0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:35.119969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5c13-b8d1-4627-b758-d46f85c8ead0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:35.120062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5c13-b8d1-4627-b758-d46f85c8ead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:35.218039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-74ce5c13-b8d1-4627-b758-d46f85c8ead0 None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:35.220220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74ce5c13-b8d1-4627-b758-d46f85c8ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:35.251933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1159/2349] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:46:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:35.259620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f41c6-6de4-4f7d-a640-a0267b8fd6be None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:35.274034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f41c6-6de4-4f7d-a640-a0267b8fd6be None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:35.278027 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:35.278935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:35.279265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:35.279603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:35.300037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:35.303241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:35.327314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1191/2350] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:46:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:46:43.172475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb4c3-a41a-4a53-8ef5-a32cc318503c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:43.174019 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb4c3-a41a-4a53-8ef5-a32cc318503c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:43.174390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb4c3-a41a-4a53-8ef5-a32cc318503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:43.212308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-437eb4c3-a41a-4a53-8ef5-a32cc318503c None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:43.214573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-437eb4c3-a41a-4a53-8ef5-a32cc318503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:43.251064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1160/2351] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:43.274189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:43.300252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e8b00-e258-4979-a3d0-6266534a2beb service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:43.316499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:43.317401 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:43.317988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:43.318463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:43.331483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:43.334005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:43.348085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1192/2352] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:46:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:46:53.204336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=20 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:53.342445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=70 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:46:59.203448 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:59.204062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:59.204275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:59.230960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:46:59.232320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:46:59.240333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1161/2353] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:46:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:46:59.260776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:46:59.274159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2013dc-0b0d-4a22-8b61-7cb78f59844c service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:46:59.279837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:46:59.280485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:46:59.280767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:46:59.281045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:46:59.291395 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:46:59.293063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:46:59.315353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1193/2354] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:47:01.606943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:01.610126 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:01.647964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4f8ee-82a5-4fca-be4d-c4f81a9ba485 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:01.652923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:01.653602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:01.653888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:01.654162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] PATH_INFO: `/v3/users/26c501f3361e46239a6a492d0858afe2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:01.655376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c501f3361e46239a6a492d0858afe2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:01.657325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:01.738803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c501f3361e46239a6a492d0858afe2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:01.739826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c501f3361e46239a6a492d0858afe2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:01.752947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c501f3361e46239a6a492d0858afe2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:01.766631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c501f3361e46239a6a492d0858afe2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:01.767671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ccf7adc-de95-4c74-b0ad-b61fb8358a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c501f3361e46239a6a492d0858afe2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:01.796491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1162/2355] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:47:01 2022] DELETE /identity/v3/users/26c501f3361e46239a6a492d0858afe2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:47:02.050839 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.093849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5242-166d-4667-8ff6-17e37e2b60b8 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.187030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.188418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.188825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.189095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] PATH_INFO: `/v3/projects/3e6c81f84dac4d9cafa3908a5896ae24` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.190437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6c81f84dac4d9cafa3908a5896ae24)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.193389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.283837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6c81f84dac4d9cafa3908a5896ae24', 'request_id': 'req-ffe9b964-acd4-471d-9800-377574b094e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:02.298791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6c81f84dac4d9cafa3908a5896ae24', 'request_id': 'req-ffe9b964-acd4-471d-9800-377574b094e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:02.314753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6c81f84dac4d9cafa3908a5896ae24', 'request_id': 'req-ffe9b964-acd4-471d-9800-377574b094e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:02.336601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] The token cache is being invalidate because project 3e6c81f84dac4d9cafa3908a5896ae24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:47:02.336910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:02.337642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ffe9b964-acd4-471d-9800-377574b094e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:02.339342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1194/2356] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:47:02 2022] DELETE /identity/v3/projects/3e6c81f84dac4d9cafa3908a5896ae24 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:47:02.345009 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.345293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.345587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.377750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:02.379008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:02.454311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1163/2357] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:47:02.525866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.526106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.526551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.550646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:02.552678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:02.563741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1195/2358] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:47:02.571973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.585142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed68bb-7ec1-4aa4-af6a-3e454b32f90b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.593915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.594609 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.594897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.595171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.596573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:47:02.596749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.598923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.620441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1164/2359] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:47:02.625234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.635493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a608b-07c9-4c9f-8be2-d31d4bc60366 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.640481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.641137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.641414 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.641718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.642404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.644323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.681762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1196/2360] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:47:02 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:47:02.689977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.704134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d25a4-d3d1-4826-bb84-8d8bc6a3fad5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.787694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:02.804281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.805062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.805474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.805792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.806513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.808570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.840530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1165/2361] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:47:02 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:47:02.846109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.860719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e360-b94a-4e4d-8868-f0a9f16b0b01 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.867030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.867955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.868338 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.868707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.869515 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.873312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.892249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1197/2362] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:47:02.898595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.910883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3d96f-8d33-4586-8715-f68040121b51 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.917467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.918493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.918779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.919062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.919847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.922470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1b5eb-bc81-4335-81ad-e86f1283201a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:02.940079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1166/2363] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:47:02.947360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:02.962867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ffa90-a316-49a2-90a8-1f421ec23d17 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:02.968910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:02.969732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:02.970005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:02.970271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] PATH_INFO: `/v3/projects/c852af8850fa400b9b7386fa64d9a1b7/users/1d55c5e709764e2188550ae6a939d8e7/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:02.995209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c852af8850fa400b9b7386fa64d9a1b7, user_id=1d55c5e709764e2188550ae6a939d8e7, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:02.997912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:03.013942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1198/2364] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:47:02 2022] PUT /identity/v3/projects/c852af8850fa400b9b7386fa64d9a1b7/users/1d55c5e709764e2188550ae6a939d8e7/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:47:03.024869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:03.025277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:03.025486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:03.057824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] MFA Rules not processed for user `1d55c5e709764e2188550ae6a939d8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:03.059027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:03.170051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1167/2365] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:47:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:47:03.184882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:03.186738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:03.207565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:03.288844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca34100-7a62-44c5-bd3c-79adeb9feee8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:03.295844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:03.296998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:03.297310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:03.297719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:03.314116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:03.316748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59972889-84b2-45a0-b01d-38fe13a2d097 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:03.327774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1199/2366] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:47:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:47:06.182716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:06.299370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31720ab9-c596-4478-94be-3609f96aba33 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:06.307057 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:06.308093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:06.308471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:06.308817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] PATH_INFO: `/v3/users/1d55c5e709764e2188550ae6a939d8e7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:06.310865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d55c5e709764e2188550ae6a939d8e7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:06.313797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:06.383999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d55c5e709764e2188550ae6a939d8e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:06.385074 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d55c5e709764e2188550ae6a939d8e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:06.398163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d55c5e709764e2188550ae6a939d8e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:06.412612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d55c5e709764e2188550ae6a939d8e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:06.413820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d55c5e709764e2188550ae6a939d8e7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:06.416265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:06.447693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1168/2367] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:47:06 2022] DELETE /identity/v3/users/1d55c5e709764e2188550ae6a939d8e7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:47:06.452745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:06.453318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:06.453318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:06.475454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:06.477348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:06.565484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1200/2368] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:47:06.581798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:06.584210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:06.693811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028849a-c66c-4686-b093-80dd4aa8d4f4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:06.699485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:06.700654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:06.701212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:06.701768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:06.719752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:06.722901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:06.742193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1169/2369] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:47:07.146195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:07.157217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:07.173171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce6fae-df3d-4661-b579-a6353dc4e2c1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:07.183985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:07.185194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:07.185777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:07.186833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] PATH_INFO: `/v3/projects/c852af8850fa400b9b7386fa64d9a1b7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:07.190471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c852af8850fa400b9b7386fa64d9a1b7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:07.193444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:07.270273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c852af8850fa400b9b7386fa64d9a1b7', 'request_id': 'req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:07.279796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c852af8850fa400b9b7386fa64d9a1b7', 'request_id': 'req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:07.305063 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c852af8850fa400b9b7386fa64d9a1b7', 'request_id': 'req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:07.324168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] The token cache is being invalidate because project c852af8850fa400b9b7386fa64d9a1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:47:07.324480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:07.325175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9741c44-a9b2-43a1-bb8b-c8570844b8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:47:07.328283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1201/2370] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:47:07 2022] DELETE /identity/v3/projects/c852af8850fa400b9b7386fa64d9a1b7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:47:12.784106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:12.795337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f971b-76c2-4e66-b2e3-d0884bb4170a service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:12.884963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:12.887087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:12.887087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:12.887342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:12.890476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:13.253867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:13.255748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:13.269416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1170/2371] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:47:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:47:16.526863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f472f-ce81-4289-ac16-e2453f308bfe None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:16.527424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f472f-ce81-4289-ac16-e2453f308bfe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:16.527424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f472f-ce81-4289-ac16-e2453f308bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:16.553029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-586f472f-ce81-4289-ac16-e2453f308bfe None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:16.554370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-586f472f-ce81-4289-ac16-e2453f308bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:16.609253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:16.657209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1202/2372] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:47:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:47:16.677863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:47:16.680413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:16.750047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:16.790403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803878c-7d99-4cf6-86bd-735202511335 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:47:16.797886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:47:16.798715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:16.799173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:16.799608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:16.811679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:47:16.814582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:47:16.829779 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1171/2373] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:47:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:47:26.643797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:26.823101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=17 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:47:55.218137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:47:55.218967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:47:55.218967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:47:55.249577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] MFA Rules not processed for user `c362f3a63b944fbcbea080d55c8fdfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:47:55.251775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:47:55.334985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1203/2374] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:47:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:48:05.326183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=25 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:48:09.787621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:09.789995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:09.882026 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f192a57-03a2-4d15-9efb-ccc612da52b1 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:09.885589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:09.886208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:09.886481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:09.886749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:09.888691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:10.000837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:10.002923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:10.018387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1172/2375] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:14.826942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:14.828597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:14.849168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c43d9-a20d-4291-9819-bdd8d7717b29 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:14.853348 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:14.854096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:14.854437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:14.854750 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:14.857117 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:14.872901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:14.874384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:14.890120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1204/2376] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:17.064380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:17.066460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:17.090056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daca71-e90c-4290-adad-9824b4318ac6 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:17.096378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:17.097453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:17.097961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:17.098415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:17.101432 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:17.197876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:17.199548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b5275-0fe7-48a2-9938-569b8ddaf669 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:17.210304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1173/2377] 10.222.0.47 () {60 vars in 1396 bytes} [Mon Aug 1 22:48:17 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:24.880762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:48:25.224764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:25.237704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cff89-64af-4294-8b7f-b9137b6acc4c service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:25.241969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:25.242585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:25.244017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:25.244017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:25.248726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:25.322014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:25.323423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:25.333912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1205/2378] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:48:27.205308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=82 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:48:27.565249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:27.565776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:27.566315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:27.587662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] MFA Rules not processed for user `527b7873931742049db9f44104396686`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:48:27.588879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:27.596524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1174/2379] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:48:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:48:27.969238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:27.978463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef0ca8-4e4f-48ee-a040-01ae88abf183 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:27.984030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:27.984718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:27.985047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:27.985442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:27.987875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:28.000150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:28.002075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e78732-d451-4f1b-98cc-a7f1f27fd606 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:28.011747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1206/2380] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:28.412762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:28.414273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:28.511293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9271-a3ae-41b8-95e7-8b593628f4ea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:28.516895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:28.517540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:28.517851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:28.518123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:28.520030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:28.533936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:28.536411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:28.548943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1175/2381] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:33.189889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:33.190774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:33.190774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:33.211075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] MFA Rules not processed for user `a8bbeb0e65a54b1199d0fe718168af70`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:48:33.212299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:33.220012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1207/2382] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:48:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:48:33.319961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:33.341899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeabdea-9313-46ac-b1eb-8b2fb1c36ea5 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:33.349688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:33.349688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:33.350036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:33.350550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:33.353869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:33.371256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:33.372746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:33.381272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1176/2383] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:41.743663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:41.745179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:41.775227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd537b7-5872-4029-9977-aabdbabe7d2c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:41.781302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:41.782099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:41.782380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:41.782659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:41.785162 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:41.801638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:41.803763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:41.816848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1208/2384] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:43.378437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=41 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:48:48.388710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:48.392326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:48.430013 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e78cf-d161-4b8d-bff5-72f2b2bb7743 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:48.437369 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:48.438470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:48.438930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:48.439397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:48.442214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:48.515480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:48.516951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50f673-5c07-4e96-a829-805c5014744b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:48.527286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1177/2385] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:51.814269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=43 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:48:53.514342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:53.527860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9666-164c-4302-914a-e092fc5e12a0 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:53.535328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:53.536236 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:53.536738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:53.537208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:53.539277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:53.554345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:53.556419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:53.569116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1209/2386] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:48:56.828292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:56.828838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:56.828838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:56.850280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] MFA Rules not processed for user `e6d336332a65408c8a438c8f96f4415d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:48:56.851762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:56.858978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1178/2387] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:48:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:48:57.409946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:57.412320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:57.426328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:57.437242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde4301-a849-451e-a343-149dc8e369b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:57.441188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:57.441942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:57.442291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:57.442650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:57.444655 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:48:57.469920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:57.472610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:57.482911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1179/2388] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:48:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:48:57.509623 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70423-61e5-496b-92f6-3918d309c7b0 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:57.515539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:57.516206 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:57.516485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:57.516765 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] PATH_INFO: `/v3/users/c362f3a63b944fbcbea080d55c8fdfa1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:57.518646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c362f3a63b944fbcbea080d55c8fdfa1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:57.521835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:57.588553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c362f3a63b944fbcbea080d55c8fdfa1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:57.599483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c362f3a63b944fbcbea080d55c8fdfa1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:57.600331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c362f3a63b944fbcbea080d55c8fdfa1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:57.601160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c362f3a63b944fbcbea080d55c8fdfa1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:57.616495 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c362f3a63b944fbcbea080d55c8fdfa1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:57.629980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1210/2389] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:48:57 2022] DELETE /identity/v3/users/c362f3a63b944fbcbea080d55c8fdfa1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:48:57.844352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:48:57.861627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec060-962e-4f23-bbc8-81272f77f72f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:48:57.959974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:48:57.960764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:48:57.961254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:48:57.961579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] PATH_INFO: `/v3/projects/3b04d30f1426408ba5c8fa0238982089` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:48:57.963170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b04d30f1426408ba5c8fa0238982089)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:48:57.965400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:48:58.030838 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b04d30f1426408ba5c8fa0238982089', 'request_id': 'req-ea2da309-ecc5-449b-8bd4-27ad65d55197'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:58.046698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b04d30f1426408ba5c8fa0238982089', 'request_id': 'req-ea2da309-ecc5-449b-8bd4-27ad65d55197'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:58.054138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b04d30f1426408ba5c8fa0238982089', 'request_id': 'req-ea2da309-ecc5-449b-8bd4-27ad65d55197'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:58.071981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] The token cache is being invalidate because project 3b04d30f1426408ba5c8fa0238982089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:48:58.072277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:58.072988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:48:58.074965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1180/2390] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:48:57 2022] DELETE /identity/v3/projects/3b04d30f1426408ba5c8fa0238982089 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:49:06.006784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:06.008974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:06.092808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:06.099418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf23e-9343-4967-9e0d-2741c9f68aa9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:06.169777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:06.170360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:06.170646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:06.170920 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:06.173204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:06.186538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:06.188945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:06.205663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1211/2391] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:08.068937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:16.198404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=32 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:38.976243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:38.977956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:39.040744 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2da309-ecc5-449b-8bd4-27ad65d55197 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:39.044602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:39.045236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:39.045510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:39.045806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:39.047662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:39.113317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:39.114797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:39.125463 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1181/2392] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:43.074413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.076055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:43.092271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace2c02-61a0-4a71-b145-6853f75545c4 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.097644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.098706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.098987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.099264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] PATH_INFO: `/v3/users/a8bbeb0e65a54b1199d0fe718168af70` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.100478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bbeb0e65a54b1199d0fe718168af70)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.102481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:43.166993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bbeb0e65a54b1199d0fe718168af70'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.174707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bbeb0e65a54b1199d0fe718168af70'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.175303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bbeb0e65a54b1199d0fe718168af70'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.175890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bbeb0e65a54b1199d0fe718168af70'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.189040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-509e88c6-cf4d-4c13-89b7-bd7e7727247b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bbeb0e65a54b1199d0fe718168af70'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.202418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1212/2393] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:49:43 2022] DELETE /identity/v3/users/a8bbeb0e65a54b1199d0fe718168af70 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:49:43.422568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.434155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de3f5-ccad-4b40-8e00-bfdcfbb7350c service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.530598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.530598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.533282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.533282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] PATH_INFO: `/v3/projects/6a6afb01194249d6af1c369e9f109b72` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.533282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6afb01194249d6af1c369e9f109b72)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.534163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:43.588954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6afb01194249d6af1c369e9f109b72', 'request_id': 'req-fb62390b-a1ac-4f15-b429-78272305ae3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.592037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:43.598662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6afb01194249d6af1c369e9f109b72', 'request_id': 'req-fb62390b-a1ac-4f15-b429-78272305ae3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.606615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6afb01194249d6af1c369e9f109b72', 'request_id': 'req-fb62390b-a1ac-4f15-b429-78272305ae3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.619286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] The token cache is being invalidate because project 6a6afb01194249d6af1c369e9f109b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:49:43.619636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.620337 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fb62390b-a1ac-4f15-b429-78272305ae3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:49:43.622003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1182/2394] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:49:43 2022] DELETE /identity/v3/projects/6a6afb01194249d6af1c369e9f109b72 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:49:43.627331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.627655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.627881 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.648732 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:43.650146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:43.717933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1213/2395] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:43.726985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.727270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.727577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.747460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:43.748664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:43.756715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1183/2396] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:43.761169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.771602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069545d-6301-4fc0-9a43-8477e1c91eec None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.778046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.778671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.778947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.779248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.779955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:49:43.780255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.781979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:43.794647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1214/2397] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:49:43.799205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.809008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709960d-a92e-4965-a704-f6033c8378a2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.814306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.814962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.815238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.815507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.816178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.818163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:43.849235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1184/2398] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:49:43 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:49:43.853913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.872585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b76a-49b1-49f6-a162-7cbe9ea8e660 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.940183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.940814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.941102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.941378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.942113 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.944091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:43.952344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:43.972266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1215/2399] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:49:43 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:49:43.976643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:43.987807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095238-e05d-4bf4-8053-d02f6de3ac13 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:43.993290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:43.993945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:43.994221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:43.994488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:43.995190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:43.996892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:44.008685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1185/2400] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:49:44.014196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:44.030858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cae14-b15a-49bf-ab85-294252e10bf9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:44.037406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:44.038061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:44.038334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:44.038601 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:44.039293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:44.040893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb22706-8f39-4110-9841-883d7fec025d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:44.053465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1216/2401] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:49:44.057877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:44.068619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314058-0464-488e-a13d-0de6640e958d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:44.073646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:44.074318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:44.074604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:44.074856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] PATH_INFO: `/v3/projects/01fc0c1edc794389a289555d518bfcbb/users/578e52fe28d54963b9450f607a5cd0ab/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:44.089897 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fc0c1edc794389a289555d518bfcbb, user_id=578e52fe28d54963b9450f607a5cd0ab, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:44.092523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:44.110575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1186/2402] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:49:44 2022] PUT /identity/v3/projects/01fc0c1edc794389a289555d518bfcbb/users/578e52fe28d54963b9450f607a5cd0ab/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:49:44.118602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215a51-057c-472d-8ca4-0ee75d80d4b7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:44.119143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215a51-057c-472d-8ca4-0ee75d80d4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:44.119579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215a51-057c-472d-8ca4-0ee75d80d4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:44.146833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ee215a51-057c-472d-8ca4-0ee75d80d4b7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:44.148221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ee215a51-057c-472d-8ca4-0ee75d80d4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:44.216420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1217/2403] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:44.226846 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:44.228751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:44.286961 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11ce3-017d-4f25-8842-a02958a35f26 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:44.290967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:44.291588 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:44.291867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:44.292135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:44.302322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:44.303851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:44.313760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1187/2404] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:49.130577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844335-db1b-4a81-b7fb-3683b96fd93c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:49.131496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844335-db1b-4a81-b7fb-3683b96fd93c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:49.131774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844335-db1b-4a81-b7fb-3683b96fd93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:49.166406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ad844335-db1b-4a81-b7fb-3683b96fd93c None None] MFA Rules not processed for user `578e52fe28d54963b9450f607a5cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:49.168222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad844335-db1b-4a81-b7fb-3683b96fd93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:49.261339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1218/2405] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:49:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:49.276246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:49.278420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:49.362914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d0d90-32e8-4d8d-9e2d-7a0222f03b0b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:49.370070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:49.370880 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:49.371158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:49.371427 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:49.381531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:49.383291 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1255bf1-18fc-4c69-8b77-8ff536fb4b43 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:49.393899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1188/2406] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:51.216079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7131-6106-4810-a0b3-c0b01536454f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:51.216665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7131-6106-4810-a0b3-c0b01536454f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:51.216665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7131-6106-4810-a0b3-c0b01536454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:51.237123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-58aa7131-6106-4810-a0b3-c0b01536454f None None] MFA Rules not processed for user `578e52fe28d54963b9450f607a5cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:51.238397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-58aa7131-6106-4810-a0b3-c0b01536454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:51.245661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1219/2407] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:49:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:49:54.485854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbad3-7129-43da-bbbf-59a41282bf15 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.490054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbad3-7129-43da-bbbf-59a41282bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.490362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbad3-7129-43da-bbbf-59a41282bf15 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.492147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1189/2408] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:49:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:49:54.498868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e048460-2ea5-4e73-8c4d-30e4f92a3d4c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.499455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e048460-2ea5-4e73-8c4d-30e4f92a3d4c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.499455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e048460-2ea5-4e73-8c4d-30e4f92a3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.523250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6e048460-2ea5-4e73-8c4d-30e4f92a3d4c None None] MFA Rules not processed for user `4c4de72ad26844d3a0dd6d8b26159bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:54.524567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e048460-2ea5-4e73-8c4d-30e4f92a3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:54.531645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1220/2409] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:49:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:54.546049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fbad3-7129-43da-bbbf-59a41282bf15 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:54.553718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:54.568749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fbad3-7129-43da-bbbf-59a41282bf15 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:54.573364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:54.574201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.574576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.575486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.584112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:54.585691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fa571-51a8-47c4-9a9e-15eeb28d9f67 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:54.596105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1190/2410] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:54.725449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b242a-aaaa-49b9-aad8-6e102be5f955 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.725947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b242a-aaaa-49b9-aad8-6e102be5f955 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.726102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b242a-aaaa-49b9-aad8-6e102be5f955 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.727538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1221/2411] 10.222.0.47 () {54 vars in 921 bytes} [Mon Aug 1 22:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 22:49:54.738537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b557cc-fcdd-4140-9066-c0ad4e37bd1c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.738833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b557cc-fcdd-4140-9066-c0ad4e37bd1c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.739107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b557cc-fcdd-4140-9066-c0ad4e37bd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.773453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-92b557cc-fcdd-4140-9066-c0ad4e37bd1c None None] MFA Rules not processed for user `b74c1e8e8d3643db9230f448af705d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:54.774751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92b557cc-fcdd-4140-9066-c0ad4e37bd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:54.792207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1191/2412] 10.222.0.47 () {58 vars in 1018 bytes} [Mon Aug 1 22:49:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:54.812173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b242a-aaaa-49b9-aad8-6e102be5f955 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:54.837446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b242a-aaaa-49b9-aad8-6e102be5f955 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:54.845449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:54.847086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:54.849882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:54.850507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:54.894361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:54.894361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:54.907440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1222/2413] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:49:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:49:58.253173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b1e-a096-4101-9d98-2b8edd176b35 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:58.253734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b1e-a096-4101-9d98-2b8edd176b35 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:58.253836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b1e-a096-4101-9d98-2b8edd176b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:58.276224 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3a398b1e-a096-4101-9d98-2b8edd176b35 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:49:58.277508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3a398b1e-a096-4101-9d98-2b8edd176b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:58.341812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1192/2414] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:49:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:49:58.411172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:49:58.412923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:49:58.478343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:49:58.536016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d9714-b0be-4b16-ba08-d32e3a61e418 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:49:58.547925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:49:58.548922 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:49:58.550168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:49:58.552643 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:49:58.571331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:49:58.573045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:49:58.583067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1223/2415] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:50:08.333940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=32 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:08.578799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=13 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:18.012981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755e69a-7ab3-4387-9711-17a8e59d1526 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:18.013534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755e69a-7ab3-4387-9711-17a8e59d1526 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:18.013922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755e69a-7ab3-4387-9711-17a8e59d1526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:18.044754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5755e69a-7ab3-4387-9711-17a8e59d1526 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:18.046017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5755e69a-7ab3-4387-9711-17a8e59d1526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:18.054842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1193/2416] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:50:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:18.068978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:18.082068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e049d-e6f3-467b-af78-06f173958d76 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:18.087847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:18.088483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:18.088780 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:18.090374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:18.100112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:18.101797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:18.110834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1224/2417] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:50:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:50:28.038147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:28.107230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:34.238455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:34.238999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:34.239101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:34.261212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:34.262713 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:34.271368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1194/2418] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:50:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:34.288979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:34.305706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661b8c-c09a-4ab8-a45f-39626759cb00 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:34.311217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:34.311844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:34.312118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:34.312386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:34.321192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:34.322810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:34.333652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1225/2419] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:50:37.069866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:37.072867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:37.085741 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bead5-12b8-48c0-9e3c-883eeac9d0d9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:37.090449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:37.091103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:37.091405 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:37.091674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:37.094258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:37.115966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:37.117658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:37.131634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1195/2420] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:40.601745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:40.604816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:40.677220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:40.683478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:40.709669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8f568-9450-402a-897c-9fe43514e8d3 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:40.716439 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:40.717213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:40.717678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:40.718090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] PATH_INFO: `/v3/users/e6d336332a65408c8a438c8f96f4415d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:40.719556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d336332a65408c8a438c8f96f4415d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:40.722081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:40.741199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737b11-264f-4d3b-bc05-df7b6d8320ac service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:40.749677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:40.750620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:40.752287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:40.753041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:40.756204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:40.777443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:40.779514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:40.791524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d336332a65408c8a438c8f96f4415d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:40.792527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1226/2421] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:40.794002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d336332a65408c8a438c8f96f4415d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:40.806726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d336332a65408c8a438c8f96f4415d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:40.825677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d336332a65408c8a438c8f96f4415d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:40.826650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bbfcd80-fe92-48f6-8f39-b87837b86a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d336332a65408c8a438c8f96f4415d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:40.857040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1196/2422] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:40 2022] DELETE /identity/v3/users/e6d336332a65408c8a438c8f96f4415d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:41.040629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.052773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eb4d6-5091-4a75-bacd-c2a57c7fc6d9 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.125830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.126448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.126748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.127128 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] PATH_INFO: `/v3/projects/dce176fe07434b6bbf6fa0c031c2903f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.128342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce176fe07434b6bbf6fa0c031c2903f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.130327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.181352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce176fe07434b6bbf6fa0c031c2903f', 'request_id': 'req-d59fe568-7993-4e52-b0db-cea77b912c16'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:41.190833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce176fe07434b6bbf6fa0c031c2903f', 'request_id': 'req-d59fe568-7993-4e52-b0db-cea77b912c16'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:41.198777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce176fe07434b6bbf6fa0c031c2903f', 'request_id': 'req-d59fe568-7993-4e52-b0db-cea77b912c16'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:41.212983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] The token cache is being invalidate because project dce176fe07434b6bbf6fa0c031c2903f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:41.213353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:41.214293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d59fe568-7993-4e52-b0db-cea77b912c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:41.215908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1227/2423] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:41 2022] DELETE /identity/v3/projects/dce176fe07434b6bbf6fa0c031c2903f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:41.222119 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.222629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.223006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.244979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:41.246239 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:41.308033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1197/2424] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:41.316481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.316737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.317038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.333971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:41.341846 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:41.343735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:41.357966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1228/2425] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:41.363435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.375626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77f2f-e23e-4fc4-b5d0-c979a880c086 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.381722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.382353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.382624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.382890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.383589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:50:41.383887 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.385530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.401959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1198/2426] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:50:41.407283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.417102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b7929-1970-4418-a0a0-bbe20e9192a0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.423624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.424293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.424606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.424910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.425650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.427613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.457537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1229/2427] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:41.462507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.476999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd6d2e-b1b1-4f90-8ea3-182b71047b18 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.517349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:41.540821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.541446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.541730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.541999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.542692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.544606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.568932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1199/2428] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:41.573760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.584659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8290c6-9cd9-436a-b63c-7de9a4701a02 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.589494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.590139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.590407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.590664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.591359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.592956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.603838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1230/2429] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:50:41.612223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.625927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedd495-38aa-47be-9f58-bde5609265be admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.631917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.632545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.632825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.633114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.633851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.635508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf542a9-d2d3-4ae0-9c34-14dd5e330b68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.646104 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1200/2430] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:50:41.650463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.662503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1b28a-62e0-41c0-b3ae-f2a2bead05cd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.667384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.668123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.668397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.668668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] PATH_INFO: `/v3/projects/242973178ce74013ac9a47f0743cd258/users/ae19104ca514470694f182631f31684f/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.683792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=242973178ce74013ac9a47f0743cd258, user_id=ae19104ca514470694f182631f31684f, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.686507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.701383 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1231/2431] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:41 2022] PUT /identity/v3/projects/242973178ce74013ac9a47f0743cd258/users/ae19104ca514470694f182631f31684f/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:50:41.713206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.714078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.714242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.744449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] MFA Rules not processed for user `ae19104ca514470694f182631f31684f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:41.745825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:41.822416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1201/2432] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:41.827447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.902500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadf74-458b-4fa0-a1d3-7bee82a20f6c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:41.907763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:41.908379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:41.908656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:41.908924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:41.909619 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:41.911550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:41.938755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1232/2433] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:41.943523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:41.955453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972a97-15da-4ab1-b621-b9e8b1207bb4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.020987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.021848 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.022321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.023146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.023562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.025785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.052501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1202/2434] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:41 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:42.056997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.068060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b2bf9-b299-4658-a508-54049f5161d8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.072876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.073477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.073789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.074062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.074762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.076382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.088139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1233/2435] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:42.093155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.104438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b62e05-750f-4184-8e92-969be6cbdd96 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.109292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.109988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.110262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.110555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] PATH_INFO: `/v3/projects/2717baa2c0f6491dab3a90bb7821fbae/users/06993f9163214f619e1e235783ed91b0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.129765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2717baa2c0f6491dab3a90bb7821fbae, user_id=06993f9163214f619e1e235783ed91b0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.132406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.152044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1203/2436] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:42 2022] PUT /identity/v3/projects/2717baa2c0f6491dab3a90bb7821fbae/users/06993f9163214f619e1e235783ed91b0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:42.157070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.168078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8645a-a4b5-490b-8da6-eaf8ad158bb0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.229921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.230553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.230831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.231099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.231806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.233416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1bc93-60c2-49bf-8aba-7a30b1e226cf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.243805 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1234/2437] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:42.248294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.259323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79b19-0109-440f-9c7a-d0eff294a013 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.264078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.264832 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.265213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.265491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] PATH_INFO: `/v3/projects/2717baa2c0f6491dab3a90bb7821fbae/users/06993f9163214f619e1e235783ed91b0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.267497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2717baa2c0f6491dab3a90bb7821fbae, user_id=06993f9163214f619e1e235783ed91b0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.270210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.276993 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:42.284037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1204/2438] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:42 2022] PUT /identity/v3/projects/2717baa2c0f6491dab3a90bb7821fbae/users/06993f9163214f619e1e235783ed91b0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:42.296387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.296671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.296952 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.323447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:42.327867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] MFA Rules not processed for user `06993f9163214f619e1e235783ed91b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:42.330065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:42.409215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1235/2439] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:42.422064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.423859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:42.495553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f234f-b3d7-4583-ab73-c9d7932e3906 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.500361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.501573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.502053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.502510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.520365 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.521863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.535558 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1205/2440] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:42.594339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:42.596296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:42.612232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0ea1e-6990-49e2-bff3-48417827b93e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:42.616239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:42.616935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:42.617214 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:42.617480 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:42.681712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:42.683662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01d10b-7545-4095-a360-6d6a84bc818a service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:42.697535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1236/2441] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:46.246844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:46.312672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd45e0-ffde-44fe-b0a3-5e3669ca226b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:46.317111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:46.317786 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:46.318065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:46.318334 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] PATH_INFO: `/v3/users/ae19104ca514470694f182631f31684f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:46.319591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae19104ca514470694f182631f31684f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:46.321530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:46.375966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae19104ca514470694f182631f31684f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:46.377015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae19104ca514470694f182631f31684f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:46.385700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae19104ca514470694f182631f31684f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:46.395486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae19104ca514470694f182631f31684f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:46.396162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae19104ca514470694f182631f31684f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:46.415030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1206/2442] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:46 2022] DELETE /identity/v3/users/ae19104ca514470694f182631f31684f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:46.422796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:46.423341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:46.423721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:46.444122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:46.445357 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:46.509351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1237/2443] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:46.521349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:46.522970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:46.552796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:46.588548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50608f-968e-4009-92e1-d22db7e43ffb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:46.592221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:46.592868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:46.593150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:46.593418 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:46.607285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:46.609242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:46.623275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1207/2444] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:46.923668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:46.925330 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:46.942957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f82ec-d976-4294-9e6c-7fa358bba5b0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:46.949866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:46.950272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:46.950600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:46.950914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] PATH_INFO: `/v3/projects/242973178ce74013ac9a47f0743cd258` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:46.951949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=242973178ce74013ac9a47f0743cd258)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:46.954107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:47.006197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:47.025653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242973178ce74013ac9a47f0743cd258', 'request_id': 'req-d366e7ac-242b-4cd7-a232-0dca8c6732bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.036238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242973178ce74013ac9a47f0743cd258', 'request_id': 'req-d366e7ac-242b-4cd7-a232-0dca8c6732bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.043810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242973178ce74013ac9a47f0743cd258', 'request_id': 'req-d366e7ac-242b-4cd7-a232-0dca8c6732bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.055464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] The token cache is being invalidate because project 242973178ce74013ac9a47f0743cd258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:47.055743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.056420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.058127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1238/2445] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:46 2022] DELETE /identity/v3/projects/242973178ce74013ac9a47f0743cd258 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:50:47.062796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:47.075398 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38cb33-31be-4a10-b283-bf3fa1474eec service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:47.136916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:47.137496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.137797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.138068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] PATH_INFO: `/v3/users/06993f9163214f619e1e235783ed91b0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.139235 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06993f9163214f619e1e235783ed91b0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:47.141160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:47.191212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06993f9163214f619e1e235783ed91b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.192053 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06993f9163214f619e1e235783ed91b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.200674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06993f9163214f619e1e235783ed91b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.209758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06993f9163214f619e1e235783ed91b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.210543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-902103c6-005e-43ae-be95-e66c0946d4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06993f9163214f619e1e235783ed91b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.230966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1208/2446] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:47 2022] DELETE /identity/v3/users/06993f9163214f619e1e235783ed91b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:47.522477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:47.523964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:47.539274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366e7ac-242b-4cd7-a232-0dca8c6732bf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:47.608314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:47.608967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.609280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.609577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] PATH_INFO: `/v3/projects/2717baa2c0f6491dab3a90bb7821fbae` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.610815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2717baa2c0f6491dab3a90bb7821fbae)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:47.612788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:47.661193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2717baa2c0f6491dab3a90bb7821fbae', 'request_id': 'req-6aa495ae-fe45-4863-b805-c59b11cdcda1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.672550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2717baa2c0f6491dab3a90bb7821fbae', 'request_id': 'req-6aa495ae-fe45-4863-b805-c59b11cdcda1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.683621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2717baa2c0f6491dab3a90bb7821fbae', 'request_id': 'req-6aa495ae-fe45-4863-b805-c59b11cdcda1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.695167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] The token cache is being invalidate because project 2717baa2c0f6491dab3a90bb7821fbae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:47.695445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.696101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6aa495ae-fe45-4863-b805-c59b11cdcda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:47.697656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1239/2447] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:47 2022] DELETE /identity/v3/projects/2717baa2c0f6491dab3a90bb7821fbae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:50:47.702727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.703242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.703387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.723356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:47.724424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:47.770116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:47.792863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1209/2448] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:50:47.805306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.805607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.805885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.827713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:47.828981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:47.837124 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1240/2449] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:47.841644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:47.851976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eb4e0-0fa1-4730-9d84-50b1d0156fbd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:47.856702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:47.857311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.857618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.857886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.858599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:50:47.858879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:47.860476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:47.874274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1210/2450] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:50:47.879853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:47.893795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843295f-67a1-4492-b1b7-dd703e8aefbe None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:47.898977 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:47.899627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:47.899916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:47.900187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:47.900987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:47.902988 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:47.936095 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1241/2451] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:47 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:47.940393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:47.951014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cf2bc-8e15-446e-8a19-eb86484a462b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.021490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.022562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.022918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.023379 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.024183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.026304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.052354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1211/2452] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:47 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:48.056707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.067838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da3596-cbb2-4148-9237-1c630132644f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.072924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.073546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.073847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.074150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.074880 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.076473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.088208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1242/2453] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:48.092512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.104287 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcd479-e58d-4853-82e2-2498bb88a90b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.110081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.110735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.111010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.111301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.111981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.113594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a789c10-0d97-478b-83a0-621dfd25bb87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.124412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1212/2454] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:48.128916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.140631 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b7d2d-867c-470c-b7df-e2a1615ce672 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.145879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.146553 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.146853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.147405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] PATH_INFO: `/v3/projects/faf396185d8f440a87957cec41e2b4e9/users/de71b7f3bb114e479eb97986b402a779/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.163296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:48.176555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf396185d8f440a87957cec41e2b4e9, user_id=de71b7f3bb114e479eb97986b402a779, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.181059 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.204016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1243/2455] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:48 2022] PUT /identity/v3/projects/faf396185d8f440a87957cec41e2b4e9/users/de71b7f3bb114e479eb97986b402a779/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:48.217248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.217752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.218257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.253590 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] MFA Rules not processed for user `de71b7f3bb114e479eb97986b402a779`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:48.254920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:48.345012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1213/2456] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:48.349958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.428041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f3f9a-70c9-4517-a4f6-02ec64d5f18e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.433675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.434440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.434698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.434975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.435637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.437579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.467138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1244/2457] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:48 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:48.471943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.484717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e518c-2c84-4ba7-9d32-9b8d32398122 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.531325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:48.558290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.558930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.559223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.559509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.560262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.562258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.598541 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1214/2458] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:48 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:48.603511 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.614758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d2650-3c9e-49c4-af6f-442e4f67a9b7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.619451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.620220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.620491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.620756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.621459 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.623097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.634692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1245/2459] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:48.640503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.651522 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90202316-970a-4923-9c7b-80c8098d94ed admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.656364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.657036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.657315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.657777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] PATH_INFO: `/v3/projects/80475ae7b4354b6ea99ebf467c219a23/users/663558ad01b24f7780f1af03ea8ea633/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.673852 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80475ae7b4354b6ea99ebf467c219a23, user_id=663558ad01b24f7780f1af03ea8ea633, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.676535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.692816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1215/2460] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:48 2022] PUT /identity/v3/projects/80475ae7b4354b6ea99ebf467c219a23/users/663558ad01b24f7780f1af03ea8ea633/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:48.697145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.709412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2d2e-8847-4f15-ae7c-8b49132369d2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.774796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.775316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.775599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.775994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.776777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.778484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3e03b-a16d-4b15-b12b-135956b76313 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.789788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1246/2461] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:50:48.794579 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.805378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed1fb9-6e3d-4a0c-a8f8-03d12c8e1a9e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:48.810843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:48.811509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.811781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.812047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] PATH_INFO: `/v3/projects/80475ae7b4354b6ea99ebf467c219a23/users/663558ad01b24f7780f1af03ea8ea633/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.814075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=80475ae7b4354b6ea99ebf467c219a23, user_id=663558ad01b24f7780f1af03ea8ea633, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:48.816661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:48.829473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1216/2462] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:48 2022] PUT /identity/v3/projects/80475ae7b4354b6ea99ebf467c219a23/users/663558ad01b24f7780f1af03ea8ea633/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:48.840274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:48.840602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:48.840811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:48.869522 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] MFA Rules not processed for user `663558ad01b24f7780f1af03ea8ea633`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:48.870668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:48.942085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:48.969279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1247/2463] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:48.981385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:48.983408 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:49.057220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4445c-1b1e-49f4-9d01-b87e86f792dd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:49.062128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:49.062720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:49.062990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:49.063302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:49.073923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:49.075359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acf83b-9014-4050-8c4f-18d06abc304e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:49.085907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1217/2464] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:49.736320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:49.810533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b5076-a3d2-4d89-a671-b37f2d5b6b9e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:49.815573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:49.816209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:49.816481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:49.816755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] PATH_INFO: `/v3/users/de71b7f3bb114e479eb97986b402a779` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:49.817999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] RBAC: Authorizing `identity:delete_user(user_id=de71b7f3bb114e479eb97986b402a779)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:49.819936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:49.877190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de71b7f3bb114e479eb97986b402a779'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:49.887774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de71b7f3bb114e479eb97986b402a779'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:49.888580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de71b7f3bb114e479eb97986b402a779'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:49.889276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de71b7f3bb114e479eb97986b402a779'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:49.905030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de71b7f3bb114e479eb97986b402a779'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:49.919640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1248/2465] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:49 2022] DELETE /identity/v3/users/de71b7f3bb114e479eb97986b402a779 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:49.925196 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:49.926583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:49.927076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:49.951400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:49.952688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:50.029294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1218/2466] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:50.043168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:50.045266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:50.145222 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d618e-a0ab-4941-92f0-c59287dc63af admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:50.150067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:50.150698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:50.151011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:50.151280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:50.162331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:50.164539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:50.180021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1249/2467] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:50.573574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:50.575920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:50.594963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c13ce-c102-40d4-9545-00535f0c5b6a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:50.600970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:50.601675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:50.601950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:50.602213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] PATH_INFO: `/v3/projects/faf396185d8f440a87957cec41e2b4e9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:50.603495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf396185d8f440a87957cec41e2b4e9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:50.605854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:50.655061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:50.672151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf396185d8f440a87957cec41e2b4e9', 'request_id': 'req-efbe60f1-c55a-446b-b6bd-3a803fa3d059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.681967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf396185d8f440a87957cec41e2b4e9', 'request_id': 'req-efbe60f1-c55a-446b-b6bd-3a803fa3d059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.689598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf396185d8f440a87957cec41e2b4e9', 'request_id': 'req-efbe60f1-c55a-446b-b6bd-3a803fa3d059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.701383 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] The token cache is being invalidate because project faf396185d8f440a87957cec41e2b4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:50.701696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.702373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.704014 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1219/2468] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:50 2022] DELETE /identity/v3/projects/faf396185d8f440a87957cec41e2b4e9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:50.708621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:50.718416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:50.723712 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c7d0-5090-461c-b94b-17566b8fe136 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:50.782153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:50.782918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:50.783039 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:50.783292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] PATH_INFO: `/v3/users/663558ad01b24f7780f1af03ea8ea633` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:50.784475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=663558ad01b24f7780f1af03ea8ea633)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:50.786432 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:50.839344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663558ad01b24f7780f1af03ea8ea633'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.849478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663558ad01b24f7780f1af03ea8ea633'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.850203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663558ad01b24f7780f1af03ea8ea633'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.850921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663558ad01b24f7780f1af03ea8ea633'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.868685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-10045ad8-f3de-4bf1-b939-378364ca4272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663558ad01b24f7780f1af03ea8ea633'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:50.886735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1250/2469] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:50 2022] DELETE /identity/v3/users/663558ad01b24f7780f1af03ea8ea633 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:51.239526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.242015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:51.262467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe60f1-c55a-446b-b6bd-3a803fa3d059 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.351036 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.351703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.352066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.352227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] PATH_INFO: `/v3/projects/80475ae7b4354b6ea99ebf467c219a23` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.353524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80475ae7b4354b6ea99ebf467c219a23)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.355625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.418610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80475ae7b4354b6ea99ebf467c219a23', 'request_id': 'req-891de8b5-be1a-4b70-9e9c-5485bee40ee2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:51.431283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80475ae7b4354b6ea99ebf467c219a23', 'request_id': 'req-891de8b5-be1a-4b70-9e9c-5485bee40ee2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:51.440305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80475ae7b4354b6ea99ebf467c219a23', 'request_id': 'req-891de8b5-be1a-4b70-9e9c-5485bee40ee2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:51.458716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] The token cache is being invalidate because project 80475ae7b4354b6ea99ebf467c219a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:51.458941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:51.459577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-891de8b5-be1a-4b70-9e9c-5485bee40ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:51.461151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1220/2470] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:51 2022] DELETE /identity/v3/projects/80475ae7b4354b6ea99ebf467c219a23 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:51.466668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.467256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.467536 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.493255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:51.494749 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:51.570427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1251/2471] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:51.580015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.580297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.580585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.612582 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:51.613971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:51.623790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1221/2472] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:50:51.629076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.643055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afeb406-05c4-4043-a07e-19c0608890c9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.647784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.648408 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.648707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.648981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.649713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:50:51.650013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.651642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.665179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1252/2473] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:50:51.669771 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.679100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e1a6a-da98-44d2-ba86-f70fbfaf3587 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.684269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.684892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.685303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.685602 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.686290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.688203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.717856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1222/2474] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:51 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:51.722934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.733753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3fde3-1b8e-4498-b8e5-ffdd49257189 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.742857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:51.812940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.813638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.813966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.814339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.814913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.816945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.845975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1253/2475] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:51 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:51.847280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.858986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5db64-b14f-444c-98b2-a2316241b5ac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.866056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.867086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.867455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.867730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.868591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.870851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.887436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1223/2476] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:51.889208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.900182 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9515-0c32-4542-bea5-66809fc2337d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.905791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.906424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.906684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.906960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.907700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.909359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a1dec-6947-4e67-a113-97352722684a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.922366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1254/2477] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:51.926773 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:51.937295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885afb0-eb4e-45bf-870d-afcf14a34e7d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:51.942264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:51.942954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:51.943231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:51.943501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] PATH_INFO: `/v3/projects/8283612cfb4644d1bce45a692a7d0ea7/users/bc8e45bca1094e9abd1e04a9d1717966/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:51.947341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:51.964318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8283612cfb4644d1bce45a692a7d0ea7, user_id=bc8e45bca1094e9abd1e04a9d1717966, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:51.967308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:51.986714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1224/2478] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:51 2022] PUT /identity/v3/projects/8283612cfb4644d1bce45a692a7d0ea7/users/bc8e45bca1094e9abd1e04a9d1717966/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:52.000379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.000663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.000938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.036477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] MFA Rules not processed for user `bc8e45bca1094e9abd1e04a9d1717966`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:52.037970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:52.113157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1255/2479] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:52.119195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.189877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f6171-8f08-4b28-98a6-7171ed95e99d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.194994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.195670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.195951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.196227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.196910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.198906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.229523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1225/2480] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:52 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:52.233906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.247203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2bc51-578b-4ea3-9b1f-ae425b40d471 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.310989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.311667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.311947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.312235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.312936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.314949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.337809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1256/2481] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:52 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:52.342476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.355564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9caf1-c053-4254-85d2-1e7b130a5555 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.363673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.364505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.364781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.365049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.365791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.367403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.379056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1226/2482] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:52.383603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.394211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3b97c-d5f9-46cb-8bf3-89efa2c2a954 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.399157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.399825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.400098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.400366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] PATH_INFO: `/v3/projects/f9cea257193b4c069086997dc7a31847/users/c948e619b7a74496a5fce0ef97ee3096/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.403833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:52.421879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cea257193b4c069086997dc7a31847, user_id=c948e619b7a74496a5fce0ef97ee3096, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.427683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.448035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1257/2483] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:52 2022] PUT /identity/v3/projects/f9cea257193b4c069086997dc7a31847/users/c948e619b7a74496a5fce0ef97ee3096/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:52.452855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.463086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281e08-dd79-4a70-89c5-0e57daa4fccd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.527389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.527934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.528210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.528484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.529313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.530933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d45a-b4ec-486b-a67b-f12fc006d99b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.541681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1227/2484] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:52.546211 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.558179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dfdb-6f63-460a-a8c0-ee08d305e7e4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.563855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.564543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.564819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.565089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] PATH_INFO: `/v3/projects/f9cea257193b4c069086997dc7a31847/users/c948e619b7a74496a5fce0ef97ee3096/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.567347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cea257193b4c069086997dc7a31847, user_id=c948e619b7a74496a5fce0ef97ee3096, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.570024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.581714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1258/2485] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:52 2022] PUT /identity/v3/projects/f9cea257193b4c069086997dc7a31847/users/c948e619b7a74496a5fce0ef97ee3096/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:52.593257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.593536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.593873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.624689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] MFA Rules not processed for user `c948e619b7a74496a5fce0ef97ee3096`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:52.625842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:52.682516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:52.710869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1228/2486] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:52.722555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:52.725123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:52.818745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6274-c1e5-40c9-85ea-305211db4795 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:52.822994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:52.823647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:52.823919 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:52.824184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:52.834068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:52.835426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83283-cd87-4811-af11-e57302de8ac9 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:52.846976 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1259/2487] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:53.240753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:53.316332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075ba4e-1dc8-4e4c-9f30-effc479ee272 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:53.321940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:53.322831 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:53.323248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:53.323675 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] PATH_INFO: `/v3/users/bc8e45bca1094e9abd1e04a9d1717966` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:53.325372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8e45bca1094e9abd1e04a9d1717966)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:53.327458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:53.378924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8e45bca1094e9abd1e04a9d1717966'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:53.379649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8e45bca1094e9abd1e04a9d1717966'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:53.386874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8e45bca1094e9abd1e04a9d1717966'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:53.395414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8e45bca1094e9abd1e04a9d1717966'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:53.396028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8e45bca1094e9abd1e04a9d1717966'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:53.413979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1229/2488] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:53 2022] DELETE /identity/v3/users/bc8e45bca1094e9abd1e04a9d1717966 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:50:53.420763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:53.421274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:53.421816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:53.442653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:53.443924 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:53.507403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1260/2489] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:53.518378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:53.519905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:53.585890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675c0f-7a7c-46bf-902a-239d75e2835f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:53.590863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:53.591498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:53.591772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:53.592039 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:53.607391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:53.608883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:53.619731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1230/2490] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:53.928254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:53.929993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:53.962926 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833593a1-92f1-4c22-b069-9d6711e2b05d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:53.968720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:53.969349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:53.969978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:53.970250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] PATH_INFO: `/v3/projects/8283612cfb4644d1bce45a692a7d0ea7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:53.971452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8283612cfb4644d1bce45a692a7d0ea7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:53.973442 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:54.031806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8283612cfb4644d1bce45a692a7d0ea7', 'request_id': 'req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.041889 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8283612cfb4644d1bce45a692a7d0ea7', 'request_id': 'req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.048223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8283612cfb4644d1bce45a692a7d0ea7', 'request_id': 'req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.055655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:54.061341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] The token cache is being invalidate because project 8283612cfb4644d1bce45a692a7d0ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:54.061741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.062669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.064722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1261/2491] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:53 2022] DELETE /identity/v3/projects/8283612cfb4644d1bce45a692a7d0ea7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:54.070413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:54.077340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:54.084615 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6bc2b-2d0e-4598-8349-11c5cde80283 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:54.157455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:54.158153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.158431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.158819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] PATH_INFO: `/v3/users/c948e619b7a74496a5fce0ef97ee3096` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.160107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c948e619b7a74496a5fce0ef97ee3096)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:54.162141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:54.215236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c948e619b7a74496a5fce0ef97ee3096'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.215945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c948e619b7a74496a5fce0ef97ee3096'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.223261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c948e619b7a74496a5fce0ef97ee3096'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.231438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c948e619b7a74496a5fce0ef97ee3096'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.232033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d75e09b4-5134-4a43-bb6e-9331aa5abaaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c948e619b7a74496a5fce0ef97ee3096'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.250305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1231/2492] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:54 2022] DELETE /identity/v3/users/c948e619b7a74496a5fce0ef97ee3096 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:54.562195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:54.563821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:54.581354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501d609-30ca-4aa1-bb88-c5c5eb4ace4c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:54.644469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:54.645116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.645387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.645688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] PATH_INFO: `/v3/projects/f9cea257193b4c069086997dc7a31847` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.646860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cea257193b4c069086997dc7a31847)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:54.648814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:54.703069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cea257193b4c069086997dc7a31847', 'request_id': 'req-f083b9c2-6eb1-4d52-bb74-102448dca3ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.710465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cea257193b4c069086997dc7a31847', 'request_id': 'req-f083b9c2-6eb1-4d52-bb74-102448dca3ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.717111 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cea257193b4c069086997dc7a31847', 'request_id': 'req-f083b9c2-6eb1-4d52-bb74-102448dca3ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.731555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] The token cache is being invalidate because project f9cea257193b4c069086997dc7a31847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:54.731766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.732525 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f083b9c2-6eb1-4d52-bb74-102448dca3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:54.734296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1262/2493] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:54 2022] DELETE /identity/v3/projects/f9cea257193b4c069086997dc7a31847 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:54.739621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.740096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.740190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.761882 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:54.763113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:54.825952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1232/2494] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:54.835188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.835475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.835755 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.858500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:54.859742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:54.871886 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1263/2495] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:54.877414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:54.888241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75afd7-6074-44bd-93f7-7b39923d7441 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:54.893681 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:54.894319 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.894578 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.894845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.895549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:50:54.895844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:54.897466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:54.910125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1233/2496] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:50:54.919706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:54.929664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa725e-3718-451d-931b-f13271b90d3e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:54.934700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:54.935317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:54.935590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:54.935949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:54.936583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:54.938496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:54.982516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1264/2497] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:54 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:54.989194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:54.999972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9daff8-49ac-4799-8400-cad22df1fb1a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.007952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:55.073043 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.073888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.074264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.074632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.075547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.078184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.103881 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1234/2498] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:54 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:50:55.109192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.121496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdeae5-4fae-4aef-9d9b-dca2596b5fae admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.127426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.128050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.128302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.128573 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.129271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.130932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.142146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1265/2499] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:55.146622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.157957 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa7c2-0bcd-4d3d-b899-4293de3fe0b0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.163631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.164293 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.164573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.164840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.165550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.167265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a358087-eebd-45f5-a435-010733e0d3f3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.178967 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1235/2500] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:55.183837 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.194945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24d1d1-0487-45e3-82ae-8c6e7c3b184d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.200392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.201060 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.201322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.201654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] PATH_INFO: `/v3/projects/53640baef52a4321870abacb58e488cc/users/99d01658db6f4c708cdb0dbb44be7f92/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.217210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=53640baef52a4321870abacb58e488cc, user_id=99d01658db6f4c708cdb0dbb44be7f92, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.220280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.242827 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1266/2501] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:55 2022] PUT /identity/v3/projects/53640baef52a4321870abacb58e488cc/users/99d01658db6f4c708cdb0dbb44be7f92/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:55.254121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.254385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.254550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.284804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] MFA Rules not processed for user `99d01658db6f4c708cdb0dbb44be7f92`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:55.286040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:55.355362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1236/2502] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:55.370101 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.372270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:55.402876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:55.442421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf157b-9cb2-4e8e-88c4-f055c03806bb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.447779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.448415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.448692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.448961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.458165 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.459624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23567c71-9ed2-43de-8042-d1ba732442a0 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.470097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1267/2503] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:55.489689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.560332 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587ac9f-8fd3-45b7-908d-01cc31416a1f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.566340 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.566996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.567261 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.567562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] PATH_INFO: `/v3/users/99d01658db6f4c708cdb0dbb44be7f92` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.568781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d01658db6f4c708cdb0dbb44be7f92)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.570748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.593814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:55.623642 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d01658db6f4c708cdb0dbb44be7f92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:55.624284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d01658db6f4c708cdb0dbb44be7f92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:55.633500 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d01658db6f4c708cdb0dbb44be7f92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:55.642371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d01658db6f4c708cdb0dbb44be7f92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:55.643130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d01658db6f4c708cdb0dbb44be7f92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:55.662703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1237/2504] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:50:55 2022] DELETE /identity/v3/users/99d01658db6f4c708cdb0dbb44be7f92 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:55.667376 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.667648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.667938 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.688256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:55.689475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:55.753296 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1268/2505] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:55.763435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:55.765195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:55.836349 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3be74-143a-43d9-bd52-07eb3ce1d9c4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:55.842052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:55.843024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:55.843491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:55.843938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:55.859249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:55.860706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6c370-a137-4be9-9962-b879422a89e3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:55.871230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1238/2506] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:50:56.175811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.177947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:56.202578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d922460-2b2f-4862-a962-427ad06414ba None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.210029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.210797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.211146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.211482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] PATH_INFO: `/v3/projects/53640baef52a4321870abacb58e488cc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.212835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53640baef52a4321870abacb58e488cc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.215040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.267440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53640baef52a4321870abacb58e488cc', 'request_id': 'req-fe07ffe6-0d17-459b-9c4e-218315fa1d62'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:56.276823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53640baef52a4321870abacb58e488cc', 'request_id': 'req-fe07ffe6-0d17-459b-9c4e-218315fa1d62'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:56.283241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53640baef52a4321870abacb58e488cc', 'request_id': 'req-fe07ffe6-0d17-459b-9c4e-218315fa1d62'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:56.297085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] The token cache is being invalidate because project 53640baef52a4321870abacb58e488cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:50:56.297373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:56.298159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe07ffe6-0d17-459b-9c4e-218315fa1d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:50:56.299828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1269/2507] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:50:56 2022] DELETE /identity/v3/projects/53640baef52a4321870abacb58e488cc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:50:56.305395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.305731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.306006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.326943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:56.328163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:56.402473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1239/2508] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:56.412394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.412806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.413160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.435940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:56.437150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:56.445478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1270/2509] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:56.450818 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.464481 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a47a24-6860-4d49-a3b7-a208ca977d44 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.471689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.472321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.472650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.472861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.473708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:50:56.473914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.475539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.489900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1240/2510] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:50:56.494656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.505361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57115b-f697-4fdb-ba0e-6816ac549744 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.511258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.511872 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.512152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.512431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.513094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.515014 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.549256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1271/2511] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:50:56 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:56.553974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.566390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2e30-828e-422b-ad01-890e4ce79d41 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.596440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:56.651024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.651667 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.651971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.652246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.653091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.654999 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.679890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1241/2512] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:50:56 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:50:56.684317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.695705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d166b1-355b-4500-ade0-a0d4f8850fa7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.700344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.700956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.701231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.701500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.702322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.703917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.715051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1272/2513] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:50:56.719200 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.730193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf964eb-85e2-4ef3-aa18-4f9784051c07 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.735132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.735756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.736032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.736304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.737048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.738700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd5173-a86f-49b9-8963-23cdd7806033 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.750527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1242/2514] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:50:56.754896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.766491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c84f-8c71-4f3a-808f-9bddef76cf07 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:56.772548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:56.773239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.773518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.773899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] PATH_INFO: `/v3/projects/50e7f7b0d3254c068b4849853b1d63d1/users/e4eb857a3166447588283de026f90ad1/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.783466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:50:56.791807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e7f7b0d3254c068b4849853b1d63d1, user_id=e4eb857a3166447588283de026f90ad1, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:56.794951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:56.810273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1273/2515] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:50:56 2022] PUT /identity/v3/projects/50e7f7b0d3254c068b4849853b1d63d1/users/e4eb857a3166447588283de026f90ad1/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:50:56.821311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f041d49-169f-4195-b664-fee72c1c5257 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:56.821952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f041d49-169f-4195-b664-fee72c1c5257 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:56.821952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f041d49-169f-4195-b664-fee72c1c5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:56.852377 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6f041d49-169f-4195-b664-fee72c1c5257 None None] MFA Rules not processed for user `e4eb857a3166447588283de026f90ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:50:56.853622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f041d49-169f-4195-b664-fee72c1c5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:56.932305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1243/2516] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:50:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:50:56.946775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:50:56.948981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:50:57.034083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add456e-9305-44bc-82a4-b88b675ac6b7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:50:57.038262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:50:57.038933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:50:57.039228 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:50:57.039579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:50:57.049501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:50:57.050970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:50:57.060679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1274/2517] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:50:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:00.709706 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659f51-e1aa-4869-92f5-9cf96c3bdbb5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:00.710300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659f51-e1aa-4869-92f5-9cf96c3bdbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:00.710300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659f51-e1aa-4869-92f5-9cf96c3bdbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:00.732524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05659f51-e1aa-4869-92f5-9cf96c3bdbb5 None None] MFA Rules not processed for user `578e52fe28d54963b9450f607a5cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:00.733790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05659f51-e1aa-4869-92f5-9cf96c3bdbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:00.793179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1244/2518] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:05.772744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:05.848692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e0894-ee5f-43e9-bfd0-50e5a6c8f5fc service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:05.853748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:05.854407 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:05.854710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:05.854982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] PATH_INFO: `/v3/users/e4eb857a3166447588283de026f90ad1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:05.856355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4eb857a3166447588283de026f90ad1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:05.858344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:05.922122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4eb857a3166447588283de026f90ad1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:05.930995 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4eb857a3166447588283de026f90ad1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:05.931636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4eb857a3166447588283de026f90ad1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:05.932373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4eb857a3166447588283de026f90ad1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:05.950174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4eb857a3166447588283de026f90ad1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:05.964966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1275/2519] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:05 2022] DELETE /identity/v3/users/e4eb857a3166447588283de026f90ad1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:05.969430 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:05.970601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:05.970895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:05.991210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:05.992564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:06.042780 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:06.079930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1245/2520] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:06.089970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:06.091561 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:06.107166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:06.174575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a1f01-1de5-44c8-87dc-57844305ef68 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:06.180561 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:06.181645 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:06.182138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:06.182632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:06.200551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:06.203246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04b4a6-c526-4bd5-b36c-d49707cdca73 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:06.218310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1276/2521] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:06.528327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:06.531494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:06.545679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729efb34-b60c-4731-97eb-0784d2117ed1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:06.552222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:06.553059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:06.553205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:06.553539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] PATH_INFO: `/v3/projects/50e7f7b0d3254c068b4849853b1d63d1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:06.554979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e7f7b0d3254c068b4849853b1d63d1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:06.556946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:06.625426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e7f7b0d3254c068b4849853b1d63d1', 'request_id': 'req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:06.637711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e7f7b0d3254c068b4849853b1d63d1', 'request_id': 'req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:06.644550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e7f7b0d3254c068b4849853b1d63d1', 'request_id': 'req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:06.669777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] The token cache is being invalidate because project 50e7f7b0d3254c068b4849853b1d63d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:06.670067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:06.670772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d0a427b0-0f18-432f-ace6-4f72ab0f0fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:06.672353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1246/2522] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:06 2022] DELETE /identity/v3/projects/50e7f7b0d3254c068b4849853b1d63d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:06.677725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:06.678054 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:06.678317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:06.770842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:06.772478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:06.928370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1277/2523] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:06.938195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:06.938484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:06.938753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:06.962348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:06.963625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:06.973709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1247/2524] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:06.981316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:06.994066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ebd5b-ceec-48f2-8642-c16c99728404 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:06.999560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.000259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.000554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.000848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.001599 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:07.001953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.003672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.018718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1278/2525] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:51:07.023525 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.035911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628d1d-3481-4935-b1f8-cb6045dc0710 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.044142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.045740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.046133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.047015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.048012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.050170 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.086518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1248/2526] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:07 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:07.093991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.106080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae5f4-2102-42ae-9497-85bc50cf3275 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.120838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:07.179410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.179927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.180231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.180503 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.181166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.183267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.208994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1279/2527] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:07 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:07.214436 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.229082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46da8bc-433c-4454-95dc-f672e29dece7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.235750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.236471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.236749 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.237024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.237766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.239738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.253371 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1249/2528] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:51:07.257737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.269403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342826c9-c69e-4d89-ab4f-215b1e39a0d6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.274592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.275385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.275654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.275915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.276734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.278463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225ffb4-75ac-41cd-acf9-8bb7a6ddfd6f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.290760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1280/2529] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:07.298796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.314142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6508c-c9b1-42c9-b945-7bb7bacd0b16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.321947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.322662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.322939 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.323216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] PATH_INFO: `/v3/projects/9d0d08943eef4e2c9bd87ed0681a2f2d/users/055598fe7df440dbbd89c4324a9fe1f0/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.350350 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0d08943eef4e2c9bd87ed0681a2f2d, user_id=055598fe7df440dbbd89c4324a9fe1f0, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.355650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.376101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1250/2530] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:07 2022] PUT /identity/v3/projects/9d0d08943eef4e2c9bd87ed0681a2f2d/users/055598fe7df440dbbd89c4324a9fe1f0/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:07.460628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.461026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.461302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.501197 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] MFA Rules not processed for user `055598fe7df440dbbd89c4324a9fe1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:07.502591 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:07.605074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1281/2531] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:07.618054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:07.620775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:07.685692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:07.713416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569d987-61bc-4fe5-b85d-a0fa3b2b695e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:07.717469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:07.718162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:07.718438 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:07.718789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:07.729483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:07.731143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:07.742595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1251/2532] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:17.601641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:17.737221 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=11 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:19.366694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:19.368259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:19.499930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead1b1d-c128-47f5-99f9-d32737df96a2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:19.506239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:19.507050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:19.507441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:19.507806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:19.510093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:19.523410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:19.525073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:19.536103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1282/2533] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:24.419474 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:24.420913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:24.484811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eafce-dccb-4727-8676-f6c0c32dfcc5 service glance] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:24.489023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:24.489677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:24.489960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:24.490229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:24.492106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:24.560351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:24.561761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:24.571962 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1252/2534] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:27.934579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:27.946141 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfa96b-31c3-4be6-bd24-d5303de8c880 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:27.956081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:27.956730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:27.957003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:27.957272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:27.959192 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:27.973505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:27.976982 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:27.993634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1283/2535] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:28.001785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.004089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:28.031785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c037d33-05c4-438f-9766-024caae40c97 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.039483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.040736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.041154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.041718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] PATH_INFO: `/v3/users/527b7873931742049db9f44104396686` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.043622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=527b7873931742049db9f44104396686)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.047155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.108488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '527b7873931742049db9f44104396686'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.109031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '527b7873931742049db9f44104396686'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.116309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '527b7873931742049db9f44104396686'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.123917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '527b7873931742049db9f44104396686'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.124619 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dc0ad08-b526-4651-927e-bfff071dd38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '527b7873931742049db9f44104396686'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.145233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1253/2536] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:28 2022] DELETE /identity/v3/users/527b7873931742049db9f44104396686 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:28.330500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.342800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eebe-5166-404b-816f-888c0d58f1b5 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.407246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.407904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.408203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.408473 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] PATH_INFO: `/v3/projects/96cf343ab1cd4658997298d18a083f29` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.409656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cf343ab1cd4658997298d18a083f29)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.411659 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.461746 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cf343ab1cd4658997298d18a083f29', 'request_id': 'req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.470778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cf343ab1cd4658997298d18a083f29', 'request_id': 'req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.473390 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:28.478876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cf343ab1cd4658997298d18a083f29', 'request_id': 'req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.492707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] The token cache is being invalidate because project 96cf343ab1cd4658997298d18a083f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:28.493003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.493823 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e1912ab4-cae8-4abb-8e7d-8dc8235e7e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:28.495396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1284/2537] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:28 2022] DELETE /identity/v3/projects/96cf343ab1cd4658997298d18a083f29 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:28.500872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.501132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.501420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.523600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:28.524628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:28.544647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:28.592498 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1254/2538] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:28.600452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.600734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.601008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.621678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:28.622983 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:28.631406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1285/2539] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:28.635901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.646376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00c5a1-db39-46be-8307-bf863bf7a367 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.651403 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.652050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.652328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.652599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.653327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:28.653641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.655313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.668546 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1255/2540] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:51:28.673018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.682970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94313a1-bee8-418e-bfcf-fbde92d89dcf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.687962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.688669 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.688945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.689249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.689967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.692058 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.732209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1286/2541] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:28 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:28.736455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.749518 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fefd2-e194-470f-9337-886de7459597 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.828755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.829393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.829711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.829975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.830659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.832600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.858785 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1256/2542] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:28 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:28.864017 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.875707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01827e-ea3e-4c3e-9727-d9f6a32606e5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.881716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.882434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.882818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.883123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.884020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.885640 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.896683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1287/2543] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:28.901231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.912406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425750-24c9-493f-95b3-c5d8a807a3c2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.918173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.919094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.919094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.920555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.920555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.921953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ec69f-1fcb-4e16-bb42-ce2d1138e4fb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.936174 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1257/2544] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:28.941094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:28.951853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4d71c-f894-41ff-8194-0db3456a5cf5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:28.956949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:28.957729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:28.958075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:28.958535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] PATH_INFO: `/v3/projects/9bce0102892344789fa8eeb849c7fb02/users/7a5cda259ff842049b056bd8a223d039/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:28.978447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bce0102892344789fa8eeb849c7fb02, user_id=7a5cda259ff842049b056bd8a223d039, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:28.981256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:28.997501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1288/2545] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:28 2022] PUT /identity/v3/projects/9bce0102892344789fa8eeb849c7fb02/users/7a5cda259ff842049b056bd8a223d039/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:29.008305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:29.009306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:29.009306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:29.043466 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] MFA Rules not processed for user `7a5cda259ff842049b056bd8a223d039`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:29.051270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:29.129963 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1258/2546] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:29.143076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:29.146467 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:29.302682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec3871-6a7b-44d3-baf2-e0be0e59dbcd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:29.309692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:29.310283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:29.310565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:29.310874 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:29.320474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:29.322394 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188758d-d7ff-4af8-8506-0a1bd9ccd757 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:29.333073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1289/2547] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:51:32.124081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:32.142801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:32.191523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c4e37-1a28-4b78-bc75-aa711d8f83c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:32.196823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:32.197512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:32.197904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:32.198176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] PATH_INFO: `/v3/users/7a5cda259ff842049b056bd8a223d039` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:32.199456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5cda259ff842049b056bd8a223d039)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:32.201416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:32.256623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5cda259ff842049b056bd8a223d039'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:32.257265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5cda259ff842049b056bd8a223d039'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:32.264207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5cda259ff842049b056bd8a223d039'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:32.272025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5cda259ff842049b056bd8a223d039'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:32.272651 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5cda259ff842049b056bd8a223d039'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:32.290010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1259/2548] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:32 2022] DELETE /identity/v3/users/7a5cda259ff842049b056bd8a223d039 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:32.294955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:32.295444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:32.295544 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:32.317057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:32.318322 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:32.359257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:32.386149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1290/2549] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:32.396328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:32.398599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:32.491718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39a8b0-39bc-46b3-b75a-39f7f68080bd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:32.496165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:32.496790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:32.497063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:32.497381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:32.508641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:32.510213 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da63374-fd94-44ce-b64e-ad1fc6adb07d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:32.520704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1260/2550] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:33.079876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.081325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:33.093975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59c780-2a70-4dd7-b797-3bbaf2784b17 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.101409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.102275 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.102555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.102964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] PATH_INFO: `/v3/projects/9bce0102892344789fa8eeb849c7fb02` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.104419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bce0102892344789fa8eeb849c7fb02)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.107218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.159533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bce0102892344789fa8eeb849c7fb02', 'request_id': 'req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:33.169036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bce0102892344789fa8eeb849c7fb02', 'request_id': 'req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:33.175448 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bce0102892344789fa8eeb849c7fb02', 'request_id': 'req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:33.188691 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] The token cache is being invalidate because project 9bce0102892344789fa8eeb849c7fb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:33.189004 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:33.189627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4aa6cd43-b1b3-4a46-aafd-b7e75dc4a7bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:33.191227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1291/2551] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:33 2022] DELETE /identity/v3/projects/9bce0102892344789fa8eeb849c7fb02 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:33.196310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.197326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.197326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.217826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:33.219055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:33.234167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:33.296868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1261/2552] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:33.306894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.307482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.307482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.329718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:33.331321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:33.340844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1292/2553] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:33.346598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.357426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda6ed9-7b99-4c34-83c3-1ea54f213928 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.362661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.363302 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.363571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.363840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.364599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:33.364899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.366557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.380301 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1262/2554] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:51:33.385076 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.396946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620f14d-b82e-4f4b-8583-ca17be7426c7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.401968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.402597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.402865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.403130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.403814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.405761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.441001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1293/2555] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:33.442115 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.453059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1d23-60ba-4c74-9165-c2b343f8d66e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.516757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.517453 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.517800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.518109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.518855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.520960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.545571 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1263/2556] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:33.550103 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.562087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ab701-9081-4287-b921-8cb3d7d42efa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.567656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.568287 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.568559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.568828 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.569531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.572462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.582683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1294/2557] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:33.587429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.598583 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633356a-96c5-4265-b529-cf6615dc030f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.603856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.604483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.604760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.605027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.605763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.607375 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977bc5-bd1c-40ae-84f3-9627affa1c45 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.619330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1264/2558] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:51:33.624637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.635588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14adf-350e-4ca0-93e7-94e8a79e6f50 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.640784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.641720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.642088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.642534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] PATH_INFO: `/v3/projects/5f825bcdd28b4ef8a2f71cdb860a0c91/users/30a867f4479b45e5b44b8aab35ed9635/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.659366 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f825bcdd28b4ef8a2f71cdb860a0c91, user_id=30a867f4479b45e5b44b8aab35ed9635, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.662073 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.676528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1295/2559] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:33 2022] PUT /identity/v3/projects/5f825bcdd28b4ef8a2f71cdb860a0c91/users/30a867f4479b45e5b44b8aab35ed9635/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:33.687665 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f2da-0449-43da-be58-57274907372d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.687960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f2da-0449-43da-be58-57274907372d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.688232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393f2da-0449-43da-be58-57274907372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.718094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2393f2da-0449-43da-be58-57274907372d None None] MFA Rules not processed for user `30a867f4479b45e5b44b8aab35ed9635`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:33.719691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2393f2da-0449-43da-be58-57274907372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:33.787314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1265/2560] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:33.792180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.858276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35c5b-a572-4f7e-b1a9-4c1f79e3a12e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.863773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.864412 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.864692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.864965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.865666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.867832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:33.873855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:33.903708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1296/2561] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:33.907756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393f2da-0449-43da-be58-57274907372d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:33.919062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393f2da-0449-43da-be58-57274907372d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:33.931079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:33.982995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:33.983677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:33.984121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:33.984491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:33.985283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:33.987400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.016321 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1266/2562] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:33 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:34.020634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.032610 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54877fa8-ce2c-4b5f-8e07-4a0680512562 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.037579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.038234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.038535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.038810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.039518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.041160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.052808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1297/2563] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:34.057509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.069585 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccc53-0223-4560-b40a-e3b966756c6b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.075521 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.076479 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.076770 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.077093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] PATH_INFO: `/v3/projects/fbc0573668544dee85689a23b6120af8/users/e22ae6e318014739a66415e4837d76e0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.094904 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc0573668544dee85689a23b6120af8, user_id=e22ae6e318014739a66415e4837d76e0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.097589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.113007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1267/2564] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:34 2022] PUT /identity/v3/projects/fbc0573668544dee85689a23b6120af8/users/e22ae6e318014739a66415e4837d76e0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:34.117332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.127962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2462de-6eeb-4deb-9d42-06d83ed21c64 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.198565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.199207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.199513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.199737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.200463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.202724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f2da-c0aa-4bc0-a263-8a5e1ccfcef4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.213664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1298/2565] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:34.218320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.229782 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a305c24-8cd4-42f0-a165-e7dfae322aaa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.234803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.235410 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.235686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.235965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] PATH_INFO: `/v3/projects/fbc0573668544dee85689a23b6120af8/users/e22ae6e318014739a66415e4837d76e0/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.237984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc0573668544dee85689a23b6120af8, user_id=e22ae6e318014739a66415e4837d76e0, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.240644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.252992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1268/2566] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:34 2022] PUT /identity/v3/projects/fbc0573668544dee85689a23b6120af8/users/e22ae6e318014739a66415e4837d76e0/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:34.263783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.264074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.264339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.294986 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] MFA Rules not processed for user `e22ae6e318014739a66415e4837d76e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:34.296488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:34.383588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1299/2567] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:34.395854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.397826 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:34.485133 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64be38-5787-4448-9f41-4e4352cb0e85 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.489842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.490472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.490769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.491034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.501698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.503178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.515784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1269/2568] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:34.598343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:34.609024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01262978-2ba7-40e6-9569-88ccea1d8687 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:34.613087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:34.613753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:34.614056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:34.614279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:34.677739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:34.680298 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:34.681884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:34.693818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1300/2569] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:43.806534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:43.860687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:43.862282 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:43.865468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:43.879711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a55ab-6e76-4d82-a44c-2a68681fcef6 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:43.885800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:43.886513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:43.886730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:43.887000 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] PATH_INFO: `/v3/users/30a867f4479b45e5b44b8aab35ed9635` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:43.888160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a867f4479b45e5b44b8aab35ed9635)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:43.890137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:43.932606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530cfe1-18a6-42f4-8545-bbdc18d84279 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:43.937860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:43.938585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:43.938899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:43.939169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] PATH_INFO: `/v3/users/055598fe7df440dbbd89c4324a9fe1f0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:43.940479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055598fe7df440dbbd89c4324a9fe1f0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:43.942589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:43.943251 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a867f4479b45e5b44b8aab35ed9635'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:43.943919 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a867f4479b45e5b44b8aab35ed9635'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:43.951816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a867f4479b45e5b44b8aab35ed9635'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:43.960910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a867f4479b45e5b44b8aab35ed9635'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:43.961530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6feb2ff-643f-4a8e-9cb9-89c1a09dc8c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a867f4479b45e5b44b8aab35ed9635'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:43.981515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1270/2570] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:43 2022] DELETE /identity/v3/users/30a867f4479b45e5b44b8aab35ed9635 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:43.986081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:43.986356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:43.986630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.000894 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055598fe7df440dbbd89c4324a9fe1f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.006129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:44.007696 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.014337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055598fe7df440dbbd89c4324a9fe1f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.014979 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055598fe7df440dbbd89c4324a9fe1f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.015796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055598fe7df440dbbd89c4324a9fe1f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.029550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f1d9fdc6-8b2a-4479-b092-7b29d928fd48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055598fe7df440dbbd89c4324a9fe1f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.041929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1301/2571] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:43 2022] DELETE /identity/v3/users/055598fe7df440dbbd89c4324a9fe1f0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:44.046370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.046652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.046923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.065826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:44.067002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.072129 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1271/2572] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:44.090241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.091625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.129849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1302/2573] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:44.139930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.141253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.150982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadb943-c8b1-4620-b0b6-878cf06a6e86 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.161647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:44.176994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:44.204321 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c776e46-c6f3-4716-b2e4-b1df7c35d2ce None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.205007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.205705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.205988 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.206258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.207929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.208545 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.208816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.209081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.215806 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.217253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.218433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.220253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.227960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1272/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:51:44.230472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1303/2575] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:51:44.583357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.584854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.591373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.593202 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.599134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698bed-85d1-42d8-bf04-35da422870fd service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.608676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b4c52-2eb9-418f-a146-6dcc92bd3d96 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.613910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.614571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.614877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.615153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] PATH_INFO: `/v3/projects/5f825bcdd28b4ef8a2f71cdb860a0c91` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.616465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f825bcdd28b4ef8a2f71cdb860a0c91)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.618431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.620940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.622177 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.622361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.622636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] PATH_INFO: `/v3/projects/9d0d08943eef4e2c9bd87ed0681a2f2d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.626460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0d08943eef4e2c9bd87ed0681a2f2d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.628824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.671990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f825bcdd28b4ef8a2f71cdb860a0c91', 'request_id': 'req-900b1bac-950d-4dbb-aefd-24702e42069e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.681847 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f825bcdd28b4ef8a2f71cdb860a0c91', 'request_id': 'req-900b1bac-950d-4dbb-aefd-24702e42069e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.690809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0d08943eef4e2c9bd87ed0681a2f2d', 'request_id': 'req-cb63b983-99a7-4c0e-bc9f-90a5223fc239'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.691384 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f825bcdd28b4ef8a2f71cdb860a0c91', 'request_id': 'req-900b1bac-950d-4dbb-aefd-24702e42069e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.701503 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0d08943eef4e2c9bd87ed0681a2f2d', 'request_id': 'req-cb63b983-99a7-4c0e-bc9f-90a5223fc239'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.708057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] The token cache is being invalidate because project 5f825bcdd28b4ef8a2f71cdb860a0c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:44.708327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0d08943eef4e2c9bd87ed0681a2f2d', 'request_id': 'req-cb63b983-99a7-4c0e-bc9f-90a5223fc239'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.708445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.708974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.710845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1304/2576] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:44 2022] DELETE /identity/v3/projects/5f825bcdd28b4ef8a2f71cdb860a0c91 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:44.714673 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.718416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.726241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] The token cache is being invalidate because project 9d0d08943eef4e2c9bd87ed0681a2f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:44.726241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.726241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb63b983-99a7-4c0e-bc9f-90a5223fc239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.728282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1273/2577] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:44 2022] DELETE /identity/v3/projects/9d0d08943eef4e2c9bd87ed0681a2f2d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:44.733530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78654e3-3b08-40ed-ac22-2a28a4dad7e1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.733899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78654e3-3b08-40ed-ac22-2a28a4dad7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.734280 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78654e3-3b08-40ed-ac22-2a28a4dad7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.755623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d78654e3-3b08-40ed-ac22-2a28a4dad7e1 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:44.756945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d78654e3-3b08-40ed-ac22-2a28a4dad7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.792785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b1bac-950d-4dbb-aefd-24702e42069e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.837506 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1274/2578] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:44.847106 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.847428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.847845 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.852143 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:44.867898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.869050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.869050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.869806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] PATH_INFO: `/v3/users/e22ae6e318014739a66415e4837d76e0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.870421 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22ae6e318014739a66415e4837d76e0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.872819 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.873322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:44.874951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:44.886864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1275/2579] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:44.891575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.901545 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70d0fe-0f70-480c-8bbf-a0a5ef0fcf03 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.907470 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.908230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.908555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.908825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.909532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:44.909865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.916022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.928188 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1276/2580] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:51:44.930798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22ae6e318014739a66415e4837d76e0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.932810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.939154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22ae6e318014739a66415e4837d76e0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.939957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22ae6e318014739a66415e4837d76e0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.940660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22ae6e318014739a66415e4837d76e0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.944079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fde76a-aba7-45d0-a4c3-e3605035a124 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:44.949886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:44.950536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:44.951569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:44.951849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:44.952685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:44.954620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:44.954829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22ae6e318014739a66415e4837d76e0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:44.967737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1305/2581] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:44 2022] DELETE /identity/v3/users/e22ae6e318014739a66415e4837d76e0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:44.989625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1277/2582] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:44 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:51:44.994455 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:44.996904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.075724 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955e264-d9bc-4d26-a5f4-d0c3944f8017 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.140927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.141684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.141875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.142154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.142834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.144801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.169042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1306/2583] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:44 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:45.173241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.183827 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f94feb-ede8-4f86-9fce-17835af7bd0c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.188929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.189577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.189864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.190169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.190874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.192472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.198234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:45.203669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1278/2584] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:45.207990 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.215651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:45.219868 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709abb02-7fd2-45a4-aa95-ed16e15dfd2e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.226385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.227339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.227974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.228582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.229743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.231867 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.243696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1307/2585] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:45.248318 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.258687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fb54f-e982-4464-a775-d71558252c0b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.264310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.264929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.265241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.265601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] PATH_INFO: `/v3/projects/77a037f71d674e339ab486c8c46f625c/users/983986550ed3446dba818dbbb5a1fe3f/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.276299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.278510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.295353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da88f0-95b7-455f-aaa3-2b4a8f9b9679 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.301139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a037f71d674e339ab486c8c46f625c, user_id=983986550ed3446dba818dbbb5a1fe3f, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.302689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.303574 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.303800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.304183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b31a0-0027-4e92-b3a8-aa7b1bf557b6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.304317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] PATH_INFO: `/v3/projects/fbc0573668544dee85689a23b6120af8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.305410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc0573668544dee85689a23b6120af8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.307519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.320266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1279/2586] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:45 2022] PUT /identity/v3/projects/77a037f71d674e339ab486c8c46f625c/users/983986550ed3446dba818dbbb5a1fe3f/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:45.332526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.332810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.333082 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.370480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] MFA Rules not processed for user `983986550ed3446dba818dbbb5a1fe3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:45.371633 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc0573668544dee85689a23b6120af8', 'request_id': 'req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:45.372336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.380053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc0573668544dee85689a23b6120af8', 'request_id': 'req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:45.386798 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc0573668544dee85689a23b6120af8', 'request_id': 'req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:45.399470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] The token cache is being invalidate because project fbc0573668544dee85689a23b6120af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:45.399721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:45.400346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d3f32fde-2cd0-4f85-9c6d-4369d1e06735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:45.402263 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1308/2587] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:45 2022] DELETE /identity/v3/projects/fbc0573668544dee85689a23b6120af8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:45.407613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778e9b-3d48-4dde-9c7c-78283b626b43 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.407918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778e9b-3d48-4dde-9c7c-78283b626b43 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.408302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778e9b-3d48-4dde-9c7c-78283b626b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.429717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c5778e9b-3d48-4dde-9c7c-78283b626b43 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:45.430981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5778e9b-3d48-4dde-9c7c-78283b626b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.451256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1280/2588] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:45.463553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.465069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.493565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1309/2589] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:45.501758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.502045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.502315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.521892 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:45.523084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.528870 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110c6e-011d-489c-878d-d08f95b29b43 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.531226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1310/2590] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:45.532726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.533391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.533700 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.534002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.535484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.544150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.545784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72309-b3e4-491b-94e9-b865ee1e8c76 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.545937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.550912 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.551671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.551980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.552257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.553057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:45.553360 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.555045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.558265 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1281/2591] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:51:45.569585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1311/2592] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:51:45.573078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.585929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0589b-9c99-4262-bb31-7fe0cbe7015e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.591839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.592471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.592735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.592998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.593705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.596010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.625345 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1282/2593] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:45.629820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.640470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34b38b-5f8a-4687-a763-ad4440431572 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.708218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:45.715419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.716075 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.716356 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.716642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.717508 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.719890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.743272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1312/2594] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:45.748708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.759762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb443-4500-4a55-98f0-bc34e6d4eb6e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.764945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.765708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.765992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.766298 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.767105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.768790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.779725 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1283/2595] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:51:45.785415 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.796704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894374-8dc4-4f62-b68f-fc1d3bdf1716 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.801788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.802406 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.802681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.802954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.803679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.805283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eff9c-34be-456c-b06f-6b4c93e8c5d7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.817253 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1313/2596] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:45.822206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:45.832992 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814cb8-4e99-481c-a6bd-d543f60e65e9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:45.839186 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:45.839972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.840233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.840488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] PATH_INFO: `/v3/projects/6c5f8b9f769045cb9305ae47d221be8f/users/e4c71f9f95d84795b7d8f901b58eaaec/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.857464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5f8b9f769045cb9305ae47d221be8f, user_id=e4c71f9f95d84795b7d8f901b58eaaec, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:45.860190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:45.875863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1284/2597] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:45 2022] PUT /identity/v3/projects/6c5f8b9f769045cb9305ae47d221be8f/users/e4c71f9f95d84795b7d8f901b58eaaec/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:45.886706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:45.887340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:45.887512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:45.918213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] MFA Rules not processed for user `e4c71f9f95d84795b7d8f901b58eaaec`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:45.919493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:45.992705 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1314/2598] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:45.997201 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.019219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:46.099478 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601ddb5-838b-4563-9965-165300e4a567 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.106914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.107592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.107946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.108282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.108996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.111078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.140185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1285/2599] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:45 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:46.144587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.157221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e4705-3fca-49d7-bc70-2206c4e71a94 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.228334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.228978 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.229281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.229549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.230285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.232316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.260252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1315/2600] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:46 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:46.263856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.275814 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0755-89e7-48a2-ba4f-6dd7c4e514a8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.281599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.282277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.282550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.282821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.283528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.285154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.297495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1286/2601] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:46.302033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.313686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f3e2-1f4a-43cf-9f2d-a649007d67fc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.319955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.320648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.320928 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.321200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] PATH_INFO: `/v3/projects/065433dde1014e9a8e12e3ad25a921a9/users/888b2f6e5d864fd798b23ed6371846c7/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.337397 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=065433dde1014e9a8e12e3ad25a921a9, user_id=888b2f6e5d864fd798b23ed6371846c7, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.340173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.355489 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1316/2602] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:46 2022] PUT /identity/v3/projects/065433dde1014e9a8e12e3ad25a921a9/users/888b2f6e5d864fd798b23ed6371846c7/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:46.359996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.372755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dd65f-1adb-453f-9deb-37b21bf445cd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.445794 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.446527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.446876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.447219 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.448037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.449762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a7ad5-ca0b-433a-b54f-545f691fb6e9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.460714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1287/2603] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:46.467340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.482639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b6555-4a61-4ad8-8a7c-6dec93cf59ed admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.488572 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.489351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.489708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.489968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] PATH_INFO: `/v3/projects/065433dde1014e9a8e12e3ad25a921a9/users/888b2f6e5d864fd798b23ed6371846c7/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.492050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065433dde1014e9a8e12e3ad25a921a9, user_id=888b2f6e5d864fd798b23ed6371846c7, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.495032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.506858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1317/2604] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:46 2022] PUT /identity/v3/projects/065433dde1014e9a8e12e3ad25a921a9/users/888b2f6e5d864fd798b23ed6371846c7/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:46.519227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.519598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.519970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.556096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] MFA Rules not processed for user `888b2f6e5d864fd798b23ed6371846c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:46.557370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:46.630932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1288/2605] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:46.645682 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:46.648049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:46.674646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:46.723677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0eb6d-472f-4fb8-a392-c05c485baae2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:46.727830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:46.728531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:46.728866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:46.729140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:46.739747 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:46.741279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e6871-69e0-4dc5-a711-4143728caf8c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:46.752018 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1318/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:48.281300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:48.290835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:48.354166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567672-d35e-4a7c-ba24-5852a95e769f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:48.359475 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:48.360137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:48.360416 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:48.360686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] PATH_INFO: `/v3/users/e4c71f9f95d84795b7d8f901b58eaaec` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:48.361932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c71f9f95d84795b7d8f901b58eaaec)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:48.363926 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:48.412874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c71f9f95d84795b7d8f901b58eaaec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:48.413527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c71f9f95d84795b7d8f901b58eaaec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:48.421076 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c71f9f95d84795b7d8f901b58eaaec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:48.429112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c71f9f95d84795b7d8f901b58eaaec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:48.429901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c71f9f95d84795b7d8f901b58eaaec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:48.448309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1289/2607] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:48 2022] DELETE /identity/v3/users/e4c71f9f95d84795b7d8f901b58eaaec => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:48.453279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:48.453901 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:48.454225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:48.480597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:48.481993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:48.546628 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1319/2608] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:48.556720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:48.558275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:48.621894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebec827-8b7d-42ee-8c81-9686e9908d7e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:48.626066 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:48.626703 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:48.626975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:48.627241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:48.637548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:48.639025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:48.649378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1290/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:48.947891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:48.949465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:48.963929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745a9bd-3afd-4cd7-afcb-550647838470 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:48.969029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:48.970089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:48.970364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:48.970629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] PATH_INFO: `/v3/projects/6c5f8b9f769045cb9305ae47d221be8f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:48.972737 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5f8b9f769045cb9305ae47d221be8f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:48.974700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:49.028310 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5f8b9f769045cb9305ae47d221be8f', 'request_id': 'req-ad84b6a6-7596-4448-a433-5f0b9ba783bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.037897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5f8b9f769045cb9305ae47d221be8f', 'request_id': 'req-ad84b6a6-7596-4448-a433-5f0b9ba783bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.044255 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5f8b9f769045cb9305ae47d221be8f', 'request_id': 'req-ad84b6a6-7596-4448-a433-5f0b9ba783bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.056199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] The token cache is being invalidate because project 6c5f8b9f769045cb9305ae47d221be8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:49.056396 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.057127 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.058760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1320/2610] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:48 2022] DELETE /identity/v3/projects/6c5f8b9f769045cb9305ae47d221be8f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:49.063035 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:49.073647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f2a-1f09-4d9d-ab6b-3f47a7efcb75 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:49.102292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:49.136935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:49.137701 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.137998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.138252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] PATH_INFO: `/v3/users/888b2f6e5d864fd798b23ed6371846c7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.139510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=888b2f6e5d864fd798b23ed6371846c7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:49.141447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:49.191983 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888b2f6e5d864fd798b23ed6371846c7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.192636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888b2f6e5d864fd798b23ed6371846c7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.201952 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888b2f6e5d864fd798b23ed6371846c7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.209907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888b2f6e5d864fd798b23ed6371846c7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.210550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffa1ec4e-ec54-4996-b2a2-947b2bad4c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888b2f6e5d864fd798b23ed6371846c7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.229414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1291/2611] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:49 2022] DELETE /identity/v3/users/888b2f6e5d864fd798b23ed6371846c7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:49.527352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:49.529008 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:49.542960 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84b6a6-7596-4448-a433-5f0b9ba783bf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:49.589688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:49.607971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:49.608632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.608910 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.609209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] PATH_INFO: `/v3/projects/065433dde1014e9a8e12e3ad25a921a9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.610411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] RBAC: Authorizing `identity:delete_project(project_id=065433dde1014e9a8e12e3ad25a921a9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:49.612342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:49.662517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065433dde1014e9a8e12e3ad25a921a9', 'request_id': 'req-97dea09f-6ebc-4efc-96c3-808b4e67012e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.673273 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065433dde1014e9a8e12e3ad25a921a9', 'request_id': 'req-97dea09f-6ebc-4efc-96c3-808b4e67012e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.679185 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065433dde1014e9a8e12e3ad25a921a9', 'request_id': 'req-97dea09f-6ebc-4efc-96c3-808b4e67012e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.691201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] The token cache is being invalidate because project 065433dde1014e9a8e12e3ad25a921a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:49.691492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.692223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-97dea09f-6ebc-4efc-96c3-808b4e67012e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:49.693916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1321/2612] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:49 2022] DELETE /identity/v3/projects/065433dde1014e9a8e12e3ad25a921a9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:49.699311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.699875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.699977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.725003 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:49.726276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:49.795335 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1292/2613] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:49.803727 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.804009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.804281 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.824955 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:49.826305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:49.834795 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1322/2614] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:49.839764 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:49.850868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515ae5d-e93c-45e8-aaf4-26b98c5a8f82 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:49.856815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:49.857414 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.857753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.859250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.859250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:49.859250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:49.860851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:49.874333 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1293/2615] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:51:49.879056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:49.888945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369b0ef-1e91-40ac-aa46-a55f77354a33 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:49.893630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:49.894257 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:49.894538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:49.894811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:49.895485 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:49.897400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:49.927518 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1323/2616] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:49 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:49.932307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:49.943238 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655e5b-e27a-42ae-86ce-5ca1266a5634 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:49.979085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:50.011614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.012290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.012544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.012835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.013511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.015708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.040068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1294/2617] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:49 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:50.044437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.055565 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce2f08-0b04-4946-8744-b8eb4dbe286b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.061245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.061897 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.062170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.062419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.063118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.064723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.075890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1324/2618] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:50.080697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.092202 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4a08c-c9e7-4c5c-9d05-c48bed59c7f6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.097311 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.097979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.098400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.098683 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.099397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.101051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a040-516f-4426-beb2-ae5a05738e33 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.115801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1295/2619] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:50.121335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.135056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b429-23d0-431f-bfc8-f7f3f59dd27c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.141193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.141947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.142200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.142469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] PATH_INFO: `/v3/projects/848aa03bf32744f898667a85d9298605/users/5a197178aec74b58a38174c694c4b767/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.160259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=848aa03bf32744f898667a85d9298605, user_id=5a197178aec74b58a38174c694c4b767, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.163083 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.180137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1325/2620] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:50 2022] PUT /identity/v3/projects/848aa03bf32744f898667a85d9298605/users/5a197178aec74b58a38174c694c4b767/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:50.191007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.191295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.191570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.223308 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] MFA Rules not processed for user `5a197178aec74b58a38174c694c4b767`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:50.224552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:50.304693 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1296/2621] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:50.309568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.376671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f58d8-bb46-46cd-aeb3-37b93119fedd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.381804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.382494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.382773 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.383071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.383740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.385855 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.403065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:50.417077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1326/2622] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:50 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:50.423634 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.437060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb464eb-e631-493c-9016-0bf4f656b5a6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.513055 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.514124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.514124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.514392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.516530 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.518757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.547570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1297/2623] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:50 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:50.553245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.566003 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8670f-f3de-4ad6-a348-b27515c06f01 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.572429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.573315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.573540 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.573937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.574734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.577036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.592135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1327/2624] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:50.597452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.610595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed069255-0b44-47f4-8e33-3ed7f159ecd7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.615875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.616604 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.616861 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.617131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] PATH_INFO: `/v3/projects/d7a547d81f9846f88dda30538b2fde11/users/9a23b3ae39dd47e99d27943cc650ee11/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.635011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a547d81f9846f88dda30538b2fde11, user_id=9a23b3ae39dd47e99d27943cc650ee11, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.637618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.654260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1298/2625] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:50 2022] PUT /identity/v3/projects/d7a547d81f9846f88dda30538b2fde11/users/9a23b3ae39dd47e99d27943cc650ee11/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:50.660526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.675616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564311cc-1374-42d7-8871-350b7589aeff admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.745001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.745736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.746021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.746325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.747109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.748964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b862-1d9b-44f1-b0ba-3278eb83bef0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.762096 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1328/2626] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:50.768288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.780252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632f956-907d-4637-a471-3e3a39e8f3be admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:50.785141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:50.785888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.786163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.786433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] PATH_INFO: `/v3/projects/d7a547d81f9846f88dda30538b2fde11/users/9a23b3ae39dd47e99d27943cc650ee11/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.788413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a547d81f9846f88dda30538b2fde11, user_id=9a23b3ae39dd47e99d27943cc650ee11, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:50.791143 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:50.794746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:50.803922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1299/2627] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:50 2022] PUT /identity/v3/projects/d7a547d81f9846f88dda30538b2fde11/users/9a23b3ae39dd47e99d27943cc650ee11/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:50.815918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:50.816205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:50.816481 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:50.849169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] MFA Rules not processed for user `9a23b3ae39dd47e99d27943cc650ee11`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:50.850427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:50.934169 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1329/2628] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:50.948635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:50.950543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:51.024450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f8ddf-37a4-49c1-87d0-c62d73c79bba admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:51.030288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:51.031289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:51.031566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:51.031835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:51.043121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:51.044705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:51.054965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1300/2629] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:51.102440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:51.104077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:51.117187 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb46c93-b55a-4787-9d46-56815edfc0e2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:51.121519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:51.122174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:51.122446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:51.122709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:51.180433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:51.181949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6552-1bde-424f-a56d-12d158d88a45 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:51.191875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1330/2630] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:52.785011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:52.786860 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:52.864078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b74f9-52a0-4a24-86ff-eb27c20c9ee2 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:52.868986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:52.869708 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:52.870336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:52.870459 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] PATH_INFO: `/v3/users/983986550ed3446dba818dbbb5a1fe3f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:52.872263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983986550ed3446dba818dbbb5a1fe3f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:52.874062 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:52.930441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983986550ed3446dba818dbbb5a1fe3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:52.931230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983986550ed3446dba818dbbb5a1fe3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:52.939231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983986550ed3446dba818dbbb5a1fe3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:52.954951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983986550ed3446dba818dbbb5a1fe3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:52.956313 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983986550ed3446dba818dbbb5a1fe3f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:52.998449 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1301/2631] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:52 2022] DELETE /identity/v3/users/983986550ed3446dba818dbbb5a1fe3f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:53.000422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:53.000900 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:53.001147 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:53.015595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:53.032527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:53.033062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:53.104782 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1331/2632] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:53.114632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:53.116073 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:53.138381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:53.179978 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa73e-3550-444a-9da7-5ec0641f4622 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:53.184101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:53.184734 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:53.185016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:53.185285 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:53.195877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:53.197424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:53.207210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1302/2633] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:53.511771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:53.513604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:53.529769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138582f-d504-4c90-935c-d904b32d7650 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:53.536718 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:53.537586 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:53.538130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:53.538424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] PATH_INFO: `/v3/projects/77a037f71d674e339ab486c8c46f625c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:53.539790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a037f71d674e339ab486c8c46f625c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:53.541907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:53.593135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a037f71d674e339ab486c8c46f625c', 'request_id': 'req-89560eed-8564-4be8-a099-a0f63765d90a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:53.603554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a037f71d674e339ab486c8c46f625c', 'request_id': 'req-89560eed-8564-4be8-a099-a0f63765d90a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:53.610142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a037f71d674e339ab486c8c46f625c', 'request_id': 'req-89560eed-8564-4be8-a099-a0f63765d90a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:53.624704 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] The token cache is being invalidate because project 77a037f71d674e339ab486c8c46f625c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:53.624992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:53.625754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89560eed-8564-4be8-a099-a0f63765d90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:53.627319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1332/2634] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:53 2022] DELETE /identity/v3/projects/77a037f71d674e339ab486c8c46f625c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:54.643728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:54.645329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:54.659315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c0e2f-bdfd-457a-a818-d14de7eeaa64 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:54.725154 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:54.725847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:54.726125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:54.726396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] PATH_INFO: `/v3/users/5a197178aec74b58a38174c694c4b767` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:54.727564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a197178aec74b58a38174c694c4b767)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:54.729686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:54.794402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a197178aec74b58a38174c694c4b767'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:54.795060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a197178aec74b58a38174c694c4b767'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:54.802093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a197178aec74b58a38174c694c4b767'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:54.810314 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a197178aec74b58a38174c694c4b767'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:54.811111 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a197178aec74b58a38174c694c4b767'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:54.830435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1303/2635] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:54 2022] DELETE /identity/v3/users/5a197178aec74b58a38174c694c4b767 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:54.835116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:54.835606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:54.835702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:54.855769 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:54.856971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:54.927250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1333/2636] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:54.941409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:54.943997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:54.995277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:55.015731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7744368-6575-4037-a078-139b9478c78c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:55.021015 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:55.021836 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:55.022160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:55.022610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:55.035830 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:55.037727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:55.049597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1304/2637] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:55.335118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:55.336615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:55.349592 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c3eb-ca46-4542-8414-a9fa0d2f5f4d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:55.355364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:55.356085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:55.356375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:55.356651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] PATH_INFO: `/v3/projects/848aa03bf32744f898667a85d9298605` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:55.357934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=848aa03bf32744f898667a85d9298605)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:55.359888 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:55.386596 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:55.411272 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848aa03bf32744f898667a85d9298605', 'request_id': 'req-0a27371e-f2d7-437f-9d18-6125ed4ee25c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.420399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848aa03bf32744f898667a85d9298605', 'request_id': 'req-0a27371e-f2d7-437f-9d18-6125ed4ee25c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.426768 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848aa03bf32744f898667a85d9298605', 'request_id': 'req-0a27371e-f2d7-437f-9d18-6125ed4ee25c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.438510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] The token cache is being invalidate because project 848aa03bf32744f898667a85d9298605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:55.438799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.439488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.441047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1334/2638] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:55 2022] DELETE /identity/v3/projects/848aa03bf32744f898667a85d9298605 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:55.445191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:55.455088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c544561-e153-4f28-ab73-3f6afcf77838 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:55.514489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:55.515138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:55.515413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:55.515682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] PATH_INFO: `/v3/users/9a23b3ae39dd47e99d27943cc650ee11` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:55.516834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a23b3ae39dd47e99d27943cc650ee11)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:55.518800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:55.571903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a23b3ae39dd47e99d27943cc650ee11'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.572687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a23b3ae39dd47e99d27943cc650ee11'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.581197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a23b3ae39dd47e99d27943cc650ee11'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.591171 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a23b3ae39dd47e99d27943cc650ee11'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.591820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3cf3b776-5448-47ab-a510-71f0c6203136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a23b3ae39dd47e99d27943cc650ee11'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:55.612058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1305/2639] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:55 2022] DELETE /identity/v3/users/9a23b3ae39dd47e99d27943cc650ee11 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:55.917071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:55.919023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:55.932167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27371e-f2d7-437f-9d18-6125ed4ee25c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.001163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.001903 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.002186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.002487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] PATH_INFO: `/v3/projects/d7a547d81f9846f88dda30538b2fde11` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.003884 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a547d81f9846f88dda30538b2fde11)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.005981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.057109 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a547d81f9846f88dda30538b2fde11', 'request_id': 'req-f377c663-4c95-40ad-b4d7-e14e16633302'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:56.068597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a547d81f9846f88dda30538b2fde11', 'request_id': 'req-f377c663-4c95-40ad-b4d7-e14e16633302'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:56.074567 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a547d81f9846f88dda30538b2fde11', 'request_id': 'req-f377c663-4c95-40ad-b4d7-e14e16633302'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:56.086877 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] The token cache is being invalidate because project d7a547d81f9846f88dda30538b2fde11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:56.087163 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:56.087914 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:56.090251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1335/2640] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:55 2022] DELETE /identity/v3/projects/d7a547d81f9846f88dda30538b2fde11 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:56.095973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fe4b-2778-4f61-80ac-bd7552d7680a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.096473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fe4b-2778-4f61-80ac-bd7552d7680a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.096611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fe4b-2778-4f61-80ac-bd7552d7680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.098844 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.100247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:56.117593 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1de4fe4b-2778-4f61-80ac-bd7552d7680a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:56.118711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1de4fe4b-2778-4f61-80ac-bd7552d7680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:56.150369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:56.164294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377c663-4c95-40ad-b4d7-e14e16633302 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.169077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.169859 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.170389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.170389 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.172279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:56.185367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1306/2641] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:56.194116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.195125 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.195406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.204819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:56.218611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:56.220527 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:56.229123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1307/2642] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:56.233932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.243964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d406c-00cf-4105-a7dd-a6475157baac None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.249031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.249686 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.249966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.250207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.250935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:51:56.251233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.252885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.255856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.257353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.267709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1308/2643] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 22:51:56.272906 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.274011 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1336/2644] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:51:56.285896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07903a03-f279-46f4-b8cf-4196bdf13401 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.298208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.298834 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.299105 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.299386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.300086 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.302063 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.336118 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1309/2645] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:51:56.337475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.348858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be001d-5231-43c6-8e4d-782d169659ec service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.554020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.554647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.554956 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.555225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.555906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.557848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.582104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1337/2646] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:51:56.586709 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.602650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fec3-02ff-4c64-9134-cfae2b5e8f36 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.608553 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.609320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.609598 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.609903 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.610623 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.612310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.624798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1310/2647] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:51:56.629066 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.639708 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08affe-1cc0-4bf6-a820-965f786e2de7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.645700 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.646335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.646613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.646887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.647618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.649226 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112c7d3-f262-4462-901a-a6326cfbd719 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.660491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1338/2648] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:56.664924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.675362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ea4f7-3fed-4329-b74f-0b3993f49155 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.680163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.680772 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.681050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.681324 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] PATH_INFO: `/v3/projects/21c0f57f75bf48668d0b08f974bf6f93/users/abc98f42930b4f7bb3a80b52ca222e63/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.697255 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c0f57f75bf48668d0b08f974bf6f93, user_id=abc98f42930b4f7bb3a80b52ca222e63, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.700165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:56.714965 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1311/2649] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:56 2022] PUT /identity/v3/projects/21c0f57f75bf48668d0b08f974bf6f93/users/abc98f42930b4f7bb3a80b52ca222e63/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:56.789304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.789677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.789885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.821948 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] MFA Rules not processed for user `abc98f42930b4f7bb3a80b52ca222e63`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:56.825531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:56.906822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1339/2650] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:56.911524 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:56.983027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4981-bd55-4557-a4a4-1868057097d4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:56.987758 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:56.988392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:56.988674 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:56.988977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:56.989692 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:56.991629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.033145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1312/2651] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:51:56 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:57.037441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.044551 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:57.049247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aba318-4bdb-4693-a463-1e6430be3a67 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.112099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.112726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.113149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.113427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.114153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.116090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.142345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1340/2652] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:51:57 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:51:57.146358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.157688 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a83ba-ce26-46f6-b4a3-dc6a043c1cf3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.162568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.163223 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.163497 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.163900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.164621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.166272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.178754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1313/2653] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:57.183482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.197433 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ec3fd-b70f-4725-8f9c-92d22642ef64 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.202535 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.203264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.203499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.203772 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] PATH_INFO: `/v3/projects/752aade8015c458eb3916cd7afcb1f14/users/97c1a0e2ad1f4a86ac1fa28d72959a10/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.227021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] RBAC: Authorizing `identity:create_grant(project_id=752aade8015c458eb3916cd7afcb1f14, user_id=97c1a0e2ad1f4a86ac1fa28d72959a10, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.229745 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.250559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1341/2654] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:57 2022] PUT /identity/v3/projects/752aade8015c458eb3916cd7afcb1f14/users/97c1a0e2ad1f4a86ac1fa28d72959a10/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:57.255243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.279294 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faff31-16b6-4caa-9cbb-ae68db5ff310 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.363520 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.364643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.365212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.365917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.367192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.370273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea8409-1e77-4a67-8335-01acba3f537f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.372823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:57.395618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.408192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1314/2655] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:51:57.412297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3f8-883a-4003-85aa-204379ea956c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.418958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.420086 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.420354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.420639 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] PATH_INFO: `/v3/projects/752aade8015c458eb3916cd7afcb1f14/users/97c1a0e2ad1f4a86ac1fa28d72959a10/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.422968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752aade8015c458eb3916cd7afcb1f14, user_id=97c1a0e2ad1f4a86ac1fa28d72959a10, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.426020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.451368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1342/2656] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:51:57 2022] PUT /identity/v3/projects/752aade8015c458eb3916cd7afcb1f14/users/97c1a0e2ad1f4a86ac1fa28d72959a10/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:57.453426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.453755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.454012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.501288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] MFA Rules not processed for user `97c1a0e2ad1f4a86ac1fa28d72959a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:57.502954 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:57.617445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1315/2657] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:51:57.631040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.633595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:57.718845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fc1f4-fdfa-4ec3-b7cc-d41c1f7f7098 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.723647 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.724757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.725048 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.725324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.736189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.737677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.752318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1343/2658] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:57.856572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:57.867584 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668747-531b-4e36-9b3d-c85dec239cc7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:57.872346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:57.873367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:57.873869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:57.874292 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:57.946477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:57.947942 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7103c7f-5bd7-4b70-adcf-a3e6a22d6e9a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:57.958901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1316/2659] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:58.280766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:58.365942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace7303-0d12-4144-b443-38dddabf92bd service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:58.370989 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:58.371664 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:58.371939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:58.372209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] PATH_INFO: `/v3/users/abc98f42930b4f7bb3a80b52ca222e63` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:58.373441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc98f42930b4f7bb3a80b52ca222e63)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:58.375559 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:58.377653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:58.440717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc98f42930b4f7bb3a80b52ca222e63'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:58.452072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc98f42930b4f7bb3a80b52ca222e63'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:58.452961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc98f42930b4f7bb3a80b52ca222e63'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:58.454334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc98f42930b4f7bb3a80b52ca222e63'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:58.477375 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc98f42930b4f7bb3a80b52ca222e63'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:58.492871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1344/2660] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:58 2022] DELETE /identity/v3/users/abc98f42930b4f7bb3a80b52ca222e63 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:58.498490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:58.499511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:58.499511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:58.527284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:51:58.528423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:58.598262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1317/2661] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:51:58.609703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:58.611354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:58.698965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa344c-fad4-4efd-8c78-5b75482f1619 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:58.705680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:58.706352 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:58.706632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:58.706909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:58.718725 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:58.721764 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:58.741815 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1345/2662] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:51:59.190618 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:59.192258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:59.207346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb21c0c-788d-4a4a-a795-456d2c20ccb7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:59.213796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:59.214687 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:59.215145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:59.215482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] PATH_INFO: `/v3/projects/21c0f57f75bf48668d0b08f974bf6f93` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:59.217195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c0f57f75bf48668d0b08f974bf6f93)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:59.220991 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:59.242367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:59.304724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c0f57f75bf48668d0b08f974bf6f93', 'request_id': 'req-41541fcd-cce8-4f24-a86a-60e7cb7a5714'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.316467 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c0f57f75bf48668d0b08f974bf6f93', 'request_id': 'req-41541fcd-cce8-4f24-a86a-60e7cb7a5714'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.323606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c0f57f75bf48668d0b08f974bf6f93', 'request_id': 'req-41541fcd-cce8-4f24-a86a-60e7cb7a5714'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.346884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] The token cache is being invalidate because project 21c0f57f75bf48668d0b08f974bf6f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:51:59.347284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.347969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.349511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1318/2663] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:59 2022] DELETE /identity/v3/projects/21c0f57f75bf48668d0b08f974bf6f93 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:51:59.355566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:59.369006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc182736-f5a0-4db4-a005-e286c939928e service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:51:59.457351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:51:59.458035 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:51:59.458306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:51:59.458569 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] PATH_INFO: `/v3/users/97c1a0e2ad1f4a86ac1fa28d72959a10` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:51:59.459743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c1a0e2ad1f4a86ac1fa28d72959a10)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:51:59.461675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:51:59.486738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:51:59.530204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c1a0e2ad1f4a86ac1fa28d72959a10'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.539087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c1a0e2ad1f4a86ac1fa28d72959a10'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.539940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c1a0e2ad1f4a86ac1fa28d72959a10'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.540899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c1a0e2ad1f4a86ac1fa28d72959a10'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.558584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4ce7098d-bf73-445b-9849-acdba83e7e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c1a0e2ad1f4a86ac1fa28d72959a10'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:51:59.571193 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1346/2664] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:51:59 2022] DELETE /identity/v3/users/97c1a0e2ad1f4a86ac1fa28d72959a10 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:51:59.897364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:51:59.899029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:51:59.925212 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541fcd-cce8-4f24-a86a-60e7cb7a5714 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.008231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.009065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.010800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.010800 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] PATH_INFO: `/v3/projects/752aade8015c458eb3916cd7afcb1f14` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.012181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] RBAC: Authorizing `identity:delete_project(project_id=752aade8015c458eb3916cd7afcb1f14)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.015433 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.080656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752aade8015c458eb3916cd7afcb1f14', 'request_id': 'req-e783428f-201d-4377-b8de-88eb4593599b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:00.092934 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752aade8015c458eb3916cd7afcb1f14', 'request_id': 'req-e783428f-201d-4377-b8de-88eb4593599b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:00.103577 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752aade8015c458eb3916cd7afcb1f14', 'request_id': 'req-e783428f-201d-4377-b8de-88eb4593599b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:00.121061 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] The token cache is being invalidate because project 752aade8015c458eb3916cd7afcb1f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:00.121526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:00.122047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e783428f-201d-4377-b8de-88eb4593599b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:00.123868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1319/2665] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:51:59 2022] DELETE /identity/v3/projects/752aade8015c458eb3916cd7afcb1f14 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:00.129658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.129949 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.130221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.158850 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:00.160092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:00.228244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1347/2666] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:00.237871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.238215 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.238395 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.267705 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:00.268958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:00.278915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1320/2667] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:00.283618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.295849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a727-3225-4bf1-bf48-9ef3fa4856c8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.301694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.302333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.302801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.302925 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.303590 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:52:00.303902 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.306074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.323315 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1348/2668] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:52:00.328977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.339664 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf0cc8-526a-40c6-9ffe-222d6a2cf71c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.346163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.346821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.347091 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.347361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.348022 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.351148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.381754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1321/2669] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:00.387602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.398378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38992-dc51-41aa-ae9e-2e23c9815358 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.464139 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.464980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.465212 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.465482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.466208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.468170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.498943 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1349/2670] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:00.503050 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.510985 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:00.516534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dcaa3-7aab-420b-99a2-d0559fc3e2b5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.525138 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.525975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.526320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.526804 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.527580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.529272 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.542060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1322/2671] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:52:00.545297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.556405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6b922-fbbb-4161-a47f-631aefcd7fb1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.562482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.563065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.563345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.563615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.564323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.566009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b87ba-7040-4258-a762-21e17b3e9eeb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.578923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1350/2672] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:00.583378 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.593256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536145d-3b75-4b5c-99bf-e4f1dbbee0aa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.598328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.599002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.599278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.599550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] PATH_INFO: `/v3/projects/02dfa0c19e7b405490a414d03a185a78/users/d530a9cb246844b98acb43d78a6d4f45/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.615389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dfa0c19e7b405490a414d03a185a78, user_id=d530a9cb246844b98acb43d78a6d4f45, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.618092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.637121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1323/2673] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:00 2022] PUT /identity/v3/projects/02dfa0c19e7b405490a414d03a185a78/users/d530a9cb246844b98acb43d78a6d4f45/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:52:00.650152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.650598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.651678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.667550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:00.691502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] MFA Rules not processed for user `d530a9cb246844b98acb43d78a6d4f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:00.692604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:00.772188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1351/2674] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:00.776694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.847601 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fbc25-b389-41a1-9657-d734c10f3e5e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.852973 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.853614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.853913 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.854189 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.854855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.856931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.888306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1324/2675] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:52:00.892447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:00.903555 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c558b-af92-4d7e-9318-78919a82a59d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:00.968327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:00.968958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:00.969620 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:00.969882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:00.970563 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:00.972939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:00.997655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1352/2676] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:00 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:01.002312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:01.014117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848bb81-34c6-4fc1-b6ff-ffedfceb3b9d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:01.020650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:01.021307 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.022120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.022120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.022977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:01.024482 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:01.036083 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1325/2677] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:01.040717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:01.051235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726f484-980b-47f3-b1f1-dd81ce526d43 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:01.056177 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:01.056853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.057134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.057410 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] PATH_INFO: `/v3/projects/4d0b30ca610c4e6e8d550fb088812758/users/da17aaf7dcdf4d86bad88b6184fa4fd6/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.074975 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0b30ca610c4e6e8d550fb088812758, user_id=da17aaf7dcdf4d86bad88b6184fa4fd6, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:01.077748 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:01.091953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1353/2678] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:01 2022] PUT /identity/v3/projects/4d0b30ca610c4e6e8d550fb088812758/users/da17aaf7dcdf4d86bad88b6184fa4fd6/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:01.096672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:01.107461 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c159b-a8f6-4d00-90ec-9bfad82e6820 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:01.172859 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:01.174176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.174176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.174176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.175109 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:01.176803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48abad0e-73c0-4277-a693-83a74527e7c8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:01.190008 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1326/2679] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:52:01.194500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:01.205933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeaeb-8032-4a6a-a870-c699ab32a9ee admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:01.211259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:01.211937 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.212217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.212493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] PATH_INFO: `/v3/projects/4d0b30ca610c4e6e8d550fb088812758/users/da17aaf7dcdf4d86bad88b6184fa4fd6/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.214848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0b30ca610c4e6e8d550fb088812758, user_id=da17aaf7dcdf4d86bad88b6184fa4fd6, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:01.217966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:01.240205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1354/2680] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:01 2022] PUT /identity/v3/projects/4d0b30ca610c4e6e8d550fb088812758/users/da17aaf7dcdf4d86bad88b6184fa4fd6/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:01.241995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.242286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.242589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.274362 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] MFA Rules not processed for user `da17aaf7dcdf4d86bad88b6184fa4fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:01.275117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:01.313411 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:01.370698 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1327/2681] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:52:01.382285 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:01.384025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:01.444122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:01.448761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aff0ab-649d-44c3-9731-f040db426ddc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:01.454491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:01.455494 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:01.455866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:01.456249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:01.574777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:01.577460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:01.593223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1355/2682] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:03.811206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:03.812680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:03.877276 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8989f7c-40c8-4d02-9852-4d35cf98fe71 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:03.880784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:03.881458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:03.881723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:03.882023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:03.884182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:03.969680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:03.971159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:03.982242 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1328/2683] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:06.100851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:06.102763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:06.116347 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060f8f9-2941-497d-923b-81c3ebf70454 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:06.123981 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:06.124634 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:06.124918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:06.125217 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:06.135787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:06.137483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee36e4-64b4-4e52-84a0-fe2c75530a6c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:06.148790 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1356/2684] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:11.447065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:11.461751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa363ea3-c892-43d5-9dd9-8a667bd1dbad service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:11.467490 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:11.468572 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:11.470354 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:11.470828 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] PATH_INFO: `/v3/users/d530a9cb246844b98acb43d78a6d4f45` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:11.482752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d530a9cb246844b98acb43d78a6d4f45)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:11.488245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:11.580145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd530a9cb246844b98acb43d78a6d4f45'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.582234 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd530a9cb246844b98acb43d78a6d4f45'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.592096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd530a9cb246844b98acb43d78a6d4f45'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.606564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd530a9cb246844b98acb43d78a6d4f45'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.607511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd530a9cb246844b98acb43d78a6d4f45'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.616888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:11.636684 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1329/2685] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:11 2022] DELETE /identity/v3/users/d530a9cb246844b98acb43d78a6d4f45 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:11.641369 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:11.641944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:11.641944 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:11.671965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:11.674062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:11.685012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:11.690468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:11.749743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1357/2686] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:11.760023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:11.761722 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:11.788894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5ce4c-baa5-4593-a5e9-606fb3f966a6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:11.794194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:11.794357 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:11.794608 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:11.794883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] PATH_INFO: `/v3/users/bdfea4c2f4f94cec9cabde2c7822b9a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:11.796144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfea4c2f4f94cec9cabde2c7822b9a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:11.798121 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:11.835766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c06a-6343-42d8-a116-e467f17db1de None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:11.843387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:11.844597 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:11.844821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:11.845241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:11.857268 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:11.858858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:11.859555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfea4c2f4f94cec9cabde2c7822b9a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.860517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfea4c2f4f94cec9cabde2c7822b9a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.869328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1358/2687] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:52:11.869750 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfea4c2f4f94cec9cabde2c7822b9a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.880178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfea4c2f4f94cec9cabde2c7822b9a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.880935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfea4c2f4f94cec9cabde2c7822b9a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:11.909717 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1330/2688] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:11 2022] DELETE /identity/v3/users/bdfea4c2f4f94cec9cabde2c7822b9a9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:12.162654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.164110 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.189176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828f3b1-a270-4c44-8502-7e780bd4f6a6 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:12.225130 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:12.258492 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:12.259304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.259677 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.260040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] PATH_INFO: `/v3/projects/02dfa0c19e7b405490a414d03a185a78` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.261666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dfa0c19e7b405490a414d03a185a78)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:12.266741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:12.268233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.269977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.286107 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c62e9-2b51-44ca-87ec-3f253fe2c341 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:12.294864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:12.295526 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.295884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.296195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] PATH_INFO: `/v3/projects/dcf006ee932d483b9cb06f83b7b86b47` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.297855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf006ee932d483b9cb06f83b7b86b47)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:12.301487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:12.334462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dfa0c19e7b405490a414d03a185a78', 'request_id': 'req-bb2cf74c-515e-40d3-a819-333edc2e0798'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.343219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dfa0c19e7b405490a414d03a185a78', 'request_id': 'req-bb2cf74c-515e-40d3-a819-333edc2e0798'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.350098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dfa0c19e7b405490a414d03a185a78', 'request_id': 'req-bb2cf74c-515e-40d3-a819-333edc2e0798'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.363020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] The token cache is being invalidate because project 02dfa0c19e7b405490a414d03a185a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:12.363276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.363957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.364853 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf006ee932d483b9cb06f83b7b86b47', 'request_id': 'req-8cf27fd8-3c13-4a37-aae3-42671d6584f0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.365711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1359/2689] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/projects/02dfa0c19e7b405490a414d03a185a78 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:12.370465 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.372284 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.374085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf006ee932d483b9cb06f83b7b86b47', 'request_id': 'req-8cf27fd8-3c13-4a37-aae3-42671d6584f0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.380452 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:12.383142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf006ee932d483b9cb06f83b7b86b47', 'request_id': 'req-8cf27fd8-3c13-4a37-aae3-42671d6584f0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.398511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] The token cache is being invalidate because project dcf006ee932d483b9cb06f83b7b86b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:12.398885 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.399945 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.401505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1331/2690] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/projects/dcf006ee932d483b9cb06f83b7b86b47 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:12.405737 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.407146 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.459579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf74c-515e-40d3-a819-333edc2e0798 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:12.486835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27fd8-3c13-4a37-aae3-42671d6584f0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:12.492614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:12.493353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.493671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.494001 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] PATH_INFO: `/v3/users/f22cf65576c6410697a0313785b3a621` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.495710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22cf65576c6410697a0313785b3a621)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:12.497811 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:12.527472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:12.528388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.528716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.529070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] PATH_INFO: `/v3/users/da17aaf7dcdf4d86bad88b6184fa4fd6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.530466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da17aaf7dcdf4d86bad88b6184fa4fd6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:12.532913 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:12.547848 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:12.561573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22cf65576c6410697a0313785b3a621'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.562483 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22cf65576c6410697a0313785b3a621'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.576368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22cf65576c6410697a0313785b3a621'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.586347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22cf65576c6410697a0313785b3a621'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.587032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-facc2017-ea9d-42ac-b58e-3a34bd201924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22cf65576c6410697a0313785b3a621'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.604422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da17aaf7dcdf4d86bad88b6184fa4fd6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.612966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1332/2691] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/users/f22cf65576c6410697a0313785b3a621 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:12.614637 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da17aaf7dcdf4d86bad88b6184fa4fd6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.615461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da17aaf7dcdf4d86bad88b6184fa4fd6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.617412 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.617723 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.617997 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.622736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da17aaf7dcdf4d86bad88b6184fa4fd6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.638510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] MFA Rules not processed for user `578e52fe28d54963b9450f607a5cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:12.639933 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.644941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da17aaf7dcdf4d86bad88b6184fa4fd6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:12.661349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1360/2692] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/users/da17aaf7dcdf4d86bad88b6184fa4fd6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:12.717396 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1333/2693] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:12.880092 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.882030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.967616 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282224-0f6e-452e-a115-75a7596b7fd7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:12.973994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:12.975747 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:12.976259 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:12.976936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:12.977219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:12.977533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] PATH_INFO: `/v3/projects/0ee7590f51f948e1b165243adea8979e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:12.979199 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee7590f51f948e1b165243adea8979e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:12.981776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.008339 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:13.052991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee7590f51f948e1b165243adea8979e', 'request_id': 'req-33fc8850-ee36-445a-9a87-bca7ed2f08c0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.064368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee7590f51f948e1b165243adea8979e', 'request_id': 'req-33fc8850-ee36-445a-9a87-bca7ed2f08c0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.071341 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee7590f51f948e1b165243adea8979e', 'request_id': 'req-33fc8850-ee36-445a-9a87-bca7ed2f08c0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.080823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70af5-6845-4e91-b203-a13420d1af4f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.085116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] The token cache is being invalidate because project 0ee7590f51f948e1b165243adea8979e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:13.085461 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.086258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-33fc8850-ee36-445a-9a87-bca7ed2f08c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.088041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1361/2694] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/projects/0ee7590f51f948e1b165243adea8979e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:13.159437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.160259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.160629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.160969 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] PATH_INFO: `/v3/projects/4d0b30ca610c4e6e8d550fb088812758` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.163274 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0b30ca610c4e6e8d550fb088812758)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.166252 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.225761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0b30ca610c4e6e8d550fb088812758', 'request_id': 'req-990d12cc-8813-40ae-94ea-d06946936db1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.236297 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0b30ca610c4e6e8d550fb088812758', 'request_id': 'req-990d12cc-8813-40ae-94ea-d06946936db1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.243915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0b30ca610c4e6e8d550fb088812758', 'request_id': 'req-990d12cc-8813-40ae-94ea-d06946936db1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.257227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] The token cache is being invalidate because project 4d0b30ca610c4e6e8d550fb088812758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:13.258185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.259169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-990d12cc-8813-40ae-94ea-d06946936db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:13.261025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1334/2695] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:12 2022] DELETE /identity/v3/projects/4d0b30ca610c4e6e8d550fb088812758 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:13.268186 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.269613 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.270081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.299411 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:13.301743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:13.321849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:13.384472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1362/2696] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:13.394359 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.394639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.394922 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.418042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:13.419534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:13.428699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1335/2697] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:13.432676 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.442758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833b78-a9a9-4087-972e-0439bca7f33d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.447429 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.448057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.448325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.448629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.449496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:52:13.449942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.451702 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.464766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1363/2698] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:52:13.469719 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.480630 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d07d-16c3-46cf-91fb-511b0c580ca3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.486198 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.487147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.487637 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.487788 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.488607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.490643 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.522769 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1336/2699] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:13.526800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.537245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97524-0fec-4b7c-b46a-806aa35361b1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.595713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.596419 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.596761 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.597098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.597806 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.599717 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.622742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1364/2700] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:13.627899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.638728 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa248-efcd-4254-b8ed-948e8b2ea1a4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.644731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.645361 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.645672 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.645929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.646716 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.648465 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.665805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1337/2701] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:13.670094 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.680588 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2594-8fa6-4941-a275-700d65bf5fdf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.686469 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.687093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.687373 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.687646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.688350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.690032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde5b25-b1f2-4cdd-b424-2545e2729ca2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.702603 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1365/2702] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:13.707348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.718685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac917e2f-1607-4585-a3c5-be897ac0c9cc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.724563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.725305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.725638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.725917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] PATH_INFO: `/v3/projects/f57496249ff94d07805ecd6e9de48afb/users/1865437c108e424baec2a24331831115/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.734372 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:13.742290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57496249ff94d07805ecd6e9de48afb, user_id=1865437c108e424baec2a24331831115, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.745998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:13.760184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1338/2703] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:13 2022] PUT /identity/v3/projects/f57496249ff94d07805ecd6e9de48afb/users/1865437c108e424baec2a24331831115/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:13.772650 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.772754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.773087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.804038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] MFA Rules not processed for user `1865437c108e424baec2a24331831115`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:13.805758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:13.881158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1366/2704] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:13.888660 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:13.966256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3fa67-504c-4e94-90c0-2f56e014ab16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:13.971738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:13.972502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:13.972775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:13.973051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:13.973755 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:13.975704 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.005284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1339/2705] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:14.010484 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.022992 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e84e-0e0b-4e05-aaf2-d9b9dde4ac46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.035327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:14.085339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.085921 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.086195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.086462 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.087089 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.089023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.119237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1367/2706] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:14 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:14.124540 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.139912 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac410ed-71a8-4071-948f-f5a340238dc8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.145010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.146040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.146317 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.146581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.147283 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.148890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.160930 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1340/2707] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:14.164980 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.175204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb9e3-727d-40eb-b601-e039266480d3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.180195 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.180812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.181085 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.181350 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.182087 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.183739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d730-e13a-410c-8df0-dc19c76cc10e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.194774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1368/2708] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:14.199980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.212241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06418629-9ad8-4334-b44f-dc975e61ff94 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.218191 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.218931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.219586 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.219878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] PATH_INFO: `/v3/projects/4f618c065ada4ce8b466b24742a946d9/users/e4d698d4234942718078174aa8f46293/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.237101 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f618c065ada4ce8b466b24742a946d9, user_id=e4d698d4234942718078174aa8f46293, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.239854 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.256006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1341/2709] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:14 2022] PUT /identity/v3/projects/4f618c065ada4ce8b466b24742a946d9/users/e4d698d4234942718078174aa8f46293/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:14.267286 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.267568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.267838 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.297812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] MFA Rules not processed for user `e4d698d4234942718078174aa8f46293`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:14.299026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:14.370965 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1369/2710] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:14.375641 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.450691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23e504-0376-4e0c-a241-6e852504cb45 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.455878 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.456596 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.456871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.457139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.458030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.459899 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.490762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1342/2711] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:14.494896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.506181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1d49-517a-407d-bd3c-bc80a887c5c4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.584019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.584763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.585042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.586080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.587267 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.588927 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.616093 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1370/2712] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:14 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:52:14.619943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.632227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5e6e1-9eda-4273-a9e7-f6cddeb72be0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.637150 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.637807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.638085 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.638388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.639102 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.640756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.652444 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1343/2713] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:14.658685 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.672069 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903241f6-7e64-4732-8e1e-1a6ac536d1be admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.682009 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.683539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.684013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.684615 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] PATH_INFO: `/v3/projects/a7635c55f4944c6d887f3248ebccd3ae/users/28fae99fc48441c8a06b0cd30f9aa5ec/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.701744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:14.708617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7635c55f4944c6d887f3248ebccd3ae, user_id=28fae99fc48441c8a06b0cd30f9aa5ec, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.712493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.729542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1371/2714] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:14 2022] PUT /identity/v3/projects/a7635c55f4944c6d887f3248ebccd3ae/users/28fae99fc48441c8a06b0cd30f9aa5ec/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:14.734884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.745718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da1775-0fcf-451a-bce5-7af16aa02001 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.756455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:14.811676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.812505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.812789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.813056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.813793 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.815462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4c936-ceb5-464d-85d8-fd945294c64a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.825763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1344/2715] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:14.829951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.841016 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007131e-492b-441e-8045-6048747e5d03 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:14.846843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:14.847519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.847796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.848046 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] PATH_INFO: `/v3/projects/a7635c55f4944c6d887f3248ebccd3ae/users/28fae99fc48441c8a06b0cd30f9aa5ec/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.850145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7635c55f4944c6d887f3248ebccd3ae, user_id=28fae99fc48441c8a06b0cd30f9aa5ec, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:14.852774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:14.864607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1372/2716] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:14 2022] PUT /identity/v3/projects/a7635c55f4944c6d887f3248ebccd3ae/users/28fae99fc48441c8a06b0cd30f9aa5ec/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:14.876886 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:14.877328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:14.877752 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:14.910515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] MFA Rules not processed for user `28fae99fc48441c8a06b0cd30f9aa5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:14.911646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:14.983207 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1345/2717] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:14.995220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:14.997057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:15.056809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6ab74-ee45-4058-95d7-2d9a4f84ed87 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:15.060753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:15.061379 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:15.061701 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:15.062174 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:15.125055 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:15.126528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:15.136116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1373/2718] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:16.726509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:16.728176 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:16.742010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4ed27-5e01-481c-be53-f808ad567b68 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:16.748320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:16.748936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:16.749216 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:16.749487 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:16.813078 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:16.814627 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:16.827295 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1346/2719] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:17.431374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:17.440636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0af184-213f-47d5-8d58-fd2619def056 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:17.444556 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:17.445194 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:17.445457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:17.445807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:17.455958 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:17.457416 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629229a-f645-451a-98f7-cd1b2f4a6f3c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:17.467571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1374/2720] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:21.405244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:21.444635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:21.474994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1548b-69c2-4661-be74-c27cbb427a3f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:21.479711 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:21.480386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:21.480792 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:21.481262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] PATH_INFO: `/v3/users/1865437c108e424baec2a24331831115` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:21.483263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1865437c108e424baec2a24331831115)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:21.485402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:21.542030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1865437c108e424baec2a24331831115'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:21.542030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1865437c108e424baec2a24331831115'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:21.552042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1865437c108e424baec2a24331831115'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:21.564568 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1865437c108e424baec2a24331831115'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:21.565288 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1865437c108e424baec2a24331831115'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:21.589948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1347/2721] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:21 2022] DELETE /identity/v3/users/1865437c108e424baec2a24331831115 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:21.595190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:21.595736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:21.595736 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:21.624134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:21.625344 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:21.687427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1375/2722] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:21.698156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:21.699735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:21.774190 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd343-35ea-4b5c-bdb9-0e42dc0e2832 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:21.778249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:21.778890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:21.779160 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:21.779431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:21.789491 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:21.790974 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:21.801874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1348/2723] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:22.096791 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:22.099107 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:22.126560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9583db3-b2d9-4453-971d-7485d0620262 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:22.132269 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:22.132929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:22.133205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:22.133475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] PATH_INFO: `/v3/projects/f57496249ff94d07805ecd6e9de48afb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:22.134730 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57496249ff94d07805ecd6e9de48afb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:22.136652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:22.149274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:22.190288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57496249ff94d07805ecd6e9de48afb', 'request_id': 'req-03cfd927-9e62-45a3-951f-d59fc8d7685b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.199849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57496249ff94d07805ecd6e9de48afb', 'request_id': 'req-03cfd927-9e62-45a3-951f-d59fc8d7685b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.207487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57496249ff94d07805ecd6e9de48afb', 'request_id': 'req-03cfd927-9e62-45a3-951f-d59fc8d7685b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.225887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] The token cache is being invalidate because project f57496249ff94d07805ecd6e9de48afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:22.226229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.226951 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.228744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1376/2724] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:22 2022] DELETE /identity/v3/projects/f57496249ff94d07805ecd6e9de48afb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:22.232867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:22.242547 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62ac43-1899-440f-a84a-06ce86697a49 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:22.291281 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:22.302883 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:22.303542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:22.303819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:22.304094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] PATH_INFO: `/v3/users/e4d698d4234942718078174aa8f46293` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:22.305286 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d698d4234942718078174aa8f46293)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:22.307262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:22.357072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d698d4234942718078174aa8f46293'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.357673 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d698d4234942718078174aa8f46293'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.365738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d698d4234942718078174aa8f46293'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.375127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d698d4234942718078174aa8f46293'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.375915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d698d4234942718078174aa8f46293'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.395669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1349/2725] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:22 2022] DELETE /identity/v3/users/e4d698d4234942718078174aa8f46293 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:22.675318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:22.676860 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:22.691904 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfd927-9e62-45a3-951f-d59fc8d7685b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:22.751751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:22.752392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:22.752667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:22.752954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] PATH_INFO: `/v3/projects/4f618c065ada4ce8b466b24742a946d9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:22.754160 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f618c065ada4ce8b466b24742a946d9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:22.756098 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:22.801404 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f618c065ada4ce8b466b24742a946d9', 'request_id': 'req-16357802-5410-4501-ba25-2038c6401b2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.809998 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f618c065ada4ce8b466b24742a946d9', 'request_id': 'req-16357802-5410-4501-ba25-2038c6401b2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.816681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f618c065ada4ce8b466b24742a946d9', 'request_id': 'req-16357802-5410-4501-ba25-2038c6401b2b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.828703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] The token cache is being invalidate because project 4f618c065ada4ce8b466b24742a946d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:22.828985 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.829719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.831370 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1377/2726] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:22 2022] DELETE /identity/v3/projects/4f618c065ada4ce8b466b24742a946d9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:22.835543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:22.837120 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:22.850808 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f424-b070-4efa-a77e-d33ddcc4a0a5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:22.911323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:22.911884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:22.912167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:22.912440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] PATH_INFO: `/v3/users/28fae99fc48441c8a06b0cd30f9aa5ec` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:22.914041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28fae99fc48441c8a06b0cd30f9aa5ec)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:22.916041 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:22.966088 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28fae99fc48441c8a06b0cd30f9aa5ec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.966891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28fae99fc48441c8a06b0cd30f9aa5ec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.974551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28fae99fc48441c8a06b0cd30f9aa5ec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.983100 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28fae99fc48441c8a06b0cd30f9aa5ec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.984245 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d1791471-3586-41a2-9891-a3447b54f0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28fae99fc48441c8a06b0cd30f9aa5ec'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:22.993030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:23.009139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1350/2727] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:52:22 2022] DELETE /identity/v3/users/28fae99fc48441c8a06b0cd30f9aa5ec => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:23.336972 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.338963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:23.352840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357802-5410-4501-ba25-2038c6401b2b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.383845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:23.413692 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.414262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.414543 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.414821 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] PATH_INFO: `/v3/projects/a7635c55f4944c6d887f3248ebccd3ae` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.416580 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7635c55f4944c6d887f3248ebccd3ae)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.418680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.465792 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7635c55f4944c6d887f3248ebccd3ae', 'request_id': 'req-53187f33-88b6-4e47-be6f-8f62e1da764e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:23.473930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7635c55f4944c6d887f3248ebccd3ae', 'request_id': 'req-53187f33-88b6-4e47-be6f-8f62e1da764e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:23.480858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7635c55f4944c6d887f3248ebccd3ae', 'request_id': 'req-53187f33-88b6-4e47-be6f-8f62e1da764e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:23.492371 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] The token cache is being invalidate because project a7635c55f4944c6d887f3248ebccd3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:52:23.492658 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:23.493293 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-53187f33-88b6-4e47-be6f-8f62e1da764e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:52:23.495120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1378/2728] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:52:23 2022] DELETE /identity/v3/projects/a7635c55f4944c6d887f3248ebccd3ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:23.500173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.500460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.500735 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.520282 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:23.521486 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:23.580842 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1351/2729] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:23.588686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.588968 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.589328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.609400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:23.610653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:23.618436 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1379/2730] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:23.623260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.635871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0fb8-0915-4653-a64e-b179c85daaca None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.642564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.642807 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.643093 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.643425 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.644181 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:52:23.644536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.646222 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.659248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1352/2731] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:52:23.664150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.673929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cfbf5-5018-4622-bc48-c8690535e273 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.678661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.679303 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.679537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.679786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.680449 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.682400 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.715775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1380/2732] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:23.716987 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.728248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d6f36-9a26-4d0e-b141-162360a1a224 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.789406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.790059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.790502 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.790676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.791277 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.793303 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.821754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1353/2733] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:23 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:52:23.823191 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.834045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263934-1a3c-4f9c-9c22-806c158c6741 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.838624 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.839297 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.839570 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.839833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.840532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.842168 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.853409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1381/2734] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:23.857889 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.868916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7391ca-1de5-4f82-a2e7-28bc6c1f45fe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.874511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.875140 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.875420 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.875690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.876423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.878116 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa02dd-d9e1-4396-ae5b-fad104a3d3bc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.889531 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1354/2735] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:23.894842 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:23.908829 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095227-7799-4e82-994b-d28b026987ae admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:23.914131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:23.914885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.915123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.915380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] PATH_INFO: `/v3/projects/8b83b190c95f4ed8901f10c9fc67b365/users/4bba292314fd415f94fb537237d9d898/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.932796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b83b190c95f4ed8901f10c9fc67b365, user_id=4bba292314fd415f94fb537237d9d898, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:23.935550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:23.951302 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1382/2736] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:23 2022] PUT /identity/v3/projects/8b83b190c95f4ed8901f10c9fc67b365/users/4bba292314fd415f94fb537237d9d898/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:52:23.956511 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b75d0-dc66-4382-90cf-1c7a17dfe1cf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:23.956781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b75d0-dc66-4382-90cf-1c7a17dfe1cf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:23.957059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b75d0-dc66-4382-90cf-1c7a17dfe1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:23.978072 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bf5b75d0-dc66-4382-90cf-1c7a17dfe1cf None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:23.979343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bf5b75d0-dc66-4382-90cf-1c7a17dfe1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:24.039856 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:24.051761 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1355/2737] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:24.061550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:24.063476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:24.123520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4117033-37b7-4c70-aa2c-d9e7a0f10fb0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:24.127190 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:24.127865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:24.128146 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:24.128413 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:24.138115 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:24.139549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:24.144537 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:24.157760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1383/2738] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:28.253810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.254422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.254422 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.284964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] MFA Rules not processed for user `4bba292314fd415f94fb537237d9d898`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:28.286203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:28.352446 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1356/2739] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:28.357152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.367577 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e95a-cdb1-43c8-9c01-6c618596afdc service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.373091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.373863 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.374209 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.374528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.375276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.377250 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.409335 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1384/2740] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:52:28 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:52:28.410551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.422165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992082c8-3f44-469d-b3e9-1b0c3642f77e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.485970 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.486565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.486844 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.487112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.487809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.489851 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.515799 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1357/2741] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:52:28 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:52:28.520030 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.533245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92d22a-e2d3-490f-96c2-229adafdf902 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.539909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.540711 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.540993 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.541256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.542033 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.543753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.556354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1385/2742] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:28.560473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.574021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5ce7a-997b-48fb-9d7d-0ffc82d55554 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.580029 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.580760 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.581038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.581306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] PATH_INFO: `/v3/projects/06ab7796b1b241628b1b854be252b709/users/f555d2973b974f388dab1791b19715a7/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.603756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ab7796b1b241628b1b854be252b709, user_id=f555d2973b974f388dab1791b19715a7, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.606603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.623515 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1358/2743] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:28 2022] PUT /identity/v3/projects/06ab7796b1b241628b1b854be252b709/users/f555d2973b974f388dab1791b19715a7/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:28.628080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.639343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fa5a0-34dd-47cc-a625-6f0fc3cb35a5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.705311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.706140 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.706423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.706696 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.707398 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.709051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b203a4-13e7-40b9-8052-b34be6dd1345 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.723698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1386/2744] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:52:28.727916 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:28.739137 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d0ea-24a5-4cfa-b292-27f1969ab949 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:28.743823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:28.744496 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:28.744778 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:28.745060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] PATH_INFO: `/v3/projects/06ab7796b1b241628b1b854be252b709/users/f555d2973b974f388dab1791b19715a7/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:28.747132 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ab7796b1b241628b1b854be252b709, user_id=f555d2973b974f388dab1791b19715a7, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:28.749896 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:28.762316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1359/2745] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:52:28 2022] PUT /identity/v3/projects/06ab7796b1b241628b1b854be252b709/users/f555d2973b974f388dab1791b19715a7/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:52:33.354688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ff211-f2d3-4cf4-8534-56c1e50f1723 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:33.355689 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ff211-f2d3-4cf4-8534-56c1e50f1723 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:33.356106 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ff211-f2d3-4cf4-8534-56c1e50f1723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:33.391890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-505ff211-f2d3-4cf4-8534-56c1e50f1723 None None] MFA Rules not processed for user `f555d2973b974f388dab1791b19715a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:33.393292 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-505ff211-f2d3-4cf4-8534-56c1e50f1723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:33.470445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1387/2746] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:52:33.493016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:52:33.495659 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:33.571457 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:33.628450 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd88ea-c104-41aa-a3a8-612061082d26 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:52:33.638089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:52:33.638748 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:33.639114 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:33.639628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:33.749406 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:52:33.751122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:52:33.761866 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1360/2747] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:52:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:52:43.467090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:43.756784 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=38 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:52:50.268114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:52:50.268657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:52:50.268657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:52:50.288582 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] MFA Rules not processed for user `4bba292314fd415f94fb537237d9d898`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:52:50.289778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:52:50.297997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1388/2748] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:52:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:53:00.282807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:53:06.729028 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:53:06.730724 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:53:06.802351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a433fb-e5c2-485b-a5cd-dcf8004a4657 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:53:06.807523 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:53:06.808230 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:53:06.808501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:53:06.808775 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:53:06.819958 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:53:06.821404 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54579b-9675-4c79-becb-c77bddc733b4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:53:06.831099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1361/2749] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:53:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:53:12.209049 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:53:12.221954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71ca91-9928-4ac0-8d8f-595d8abcbef6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:53:12.228333 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:53:12.228957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:53:12.229234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:53:12.229501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:53:12.232084 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:53:12.248205 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:53:12.250422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:53:12.263105 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1389/2750] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:53:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:53:16.497661 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:53:16.498208 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:53:16.498325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:53:16.517766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] MFA Rules not processed for user `578e52fe28d54963b9450f607a5cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:53:16.518964 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:53:16.577528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1362/2751] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:53:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:53:17.258120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:53:17.267316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54b08d-e27b-446c-a46d-b106c24cdc09 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:53:17.270997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:53:17.271602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:53:17.271871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:53:17.272136 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:53:17.273917 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:53:17.287946 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:53:17.289367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:53:17.300723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1390/2752] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:53:26.570054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=51 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:53:27.295734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=12 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:20.493576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:20.504647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e78cd9-b67e-4453-8a08-45b3565e97c6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:20.508927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:20.509936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:20.510338 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:20.510718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:20.512727 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:20.523477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:20.525065 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:20.534042 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1363/2753] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:20.847971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:20.849666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:20.879720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6883f-08b5-4f71-8f58-510de31cd809 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:20.888032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:20.889032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:20.889697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:20.890134 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:20.892203 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:20.915200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:20.917080 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:20.932896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1391/2754] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:30.448454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:30.450011 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:30.524046 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21127e-ec0e-4879-b1b5-14a8028b9322 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:30.530600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:30.531256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:30.531533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:30.531801 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] PATH_INFO: `/v3/users/578e52fe28d54963b9450f607a5cd0ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:30.533649 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578e52fe28d54963b9450f607a5cd0ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:30.536653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:30.591948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578e52fe28d54963b9450f607a5cd0ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.592550 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578e52fe28d54963b9450f607a5cd0ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.601284 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578e52fe28d54963b9450f607a5cd0ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.611249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578e52fe28d54963b9450f607a5cd0ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.612493 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4bd62eae-dc06-4d5e-8374-da01cdeb0794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578e52fe28d54963b9450f607a5cd0ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.638791 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1364/2755] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:30 2022] DELETE /identity/v3/users/578e52fe28d54963b9450f607a5cd0ab => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:30.823108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:30.836331 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc50e4-584f-4579-9ce9-932afddfeb70 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:30.913843 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:30.914496 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:30.914775 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:30.915045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] PATH_INFO: `/v3/projects/01fc0c1edc794389a289555d518bfcbb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:30.916262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fc0c1edc794389a289555d518bfcbb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:30.918215 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:30.973875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fc0c1edc794389a289555d518bfcbb', 'request_id': 'req-e2823489-b220-4ec2-9da7-e7257c7c0fa8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.986380 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fc0c1edc794389a289555d518bfcbb', 'request_id': 'req-e2823489-b220-4ec2-9da7-e7257c7c0fa8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:30.994241 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fc0c1edc794389a289555d518bfcbb', 'request_id': 'req-e2823489-b220-4ec2-9da7-e7257c7c0fa8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:31.005710 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] The token cache is being invalidate because project 01fc0c1edc794389a289555d518bfcbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:31.005947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:31.006654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2823489-b220-4ec2-9da7-e7257c7c0fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:31.008167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1392/2756] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:30 2022] DELETE /identity/v3/projects/01fc0c1edc794389a289555d518bfcbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:31.013279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.013595 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.013875 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.035037 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:31.036249 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:31.105328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1365/2757] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:31.113532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.113853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.114119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.143441 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:31.145097 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:31.158454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1393/2758] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:31.164890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.181762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6b3b5-1ecc-403a-a6c8-7dffca4b70eb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.187182 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.187802 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.188075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.188342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.189048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:54:31.189346 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.191007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.192587 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:31.205367 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1366/2759] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:54:31.209779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.220020 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fdcb8-7d72-4ede-a779-cea6d81bdc58 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.226225 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.227149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.227507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.227811 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.228738 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.231594 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.275758 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1394/2760] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:31 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:31.280134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.291312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26dcae-8883-4259-adaf-4c5c8d558e07 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.354512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.355554 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.355923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.356330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.357431 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.360258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.387426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1367/2761] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:31 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:31.393422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.408751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eda367-a724-4d88-9352-f9dc2d7e3671 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.414810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.415612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.415754 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.415957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.416652 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.418325 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.428334 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1395/2762] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:31.432534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.443058 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904bfb6-eb04-4fb3-a2dd-4b6b8116d0cf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.448426 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.449038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.449616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.449616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.450322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.451947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1fbe2-c8e5-4ae4-9410-ca2edb5b548d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.463227 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1368/2763] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:31.467600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.478833 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65cae2-fa26-4807-b041-183f3f65e65f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.484463 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.485233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.485510 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.485814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] PATH_INFO: `/v3/projects/27df862560af4235a332f6145bbfc00c/users/a03ba404d50f455db148c05bca9d5934/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.501760 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27df862560af4235a332f6145bbfc00c, user_id=a03ba404d50f455db148c05bca9d5934, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.504417 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.520447 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1396/2764] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:31 2022] PUT /identity/v3/projects/27df862560af4235a332f6145bbfc00c/users/a03ba404d50f455db148c05bca9d5934/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:54:31.531081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.531388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.531663 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.564149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] MFA Rules not processed for user `a03ba404d50f455db148c05bca9d5934`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:31.565689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:31.645876 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1369/2765] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:31.660172 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:31.661906 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:31.668963 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:31.733714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc3d2a-4d9b-4dc3-8586-f2c9f802555f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:31.737600 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:31.738239 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:31.738516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:31.738776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:31.747434 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:31.748866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12bbd-7225-420b-a595-28e42e0a6e6d service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:31.757501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1397/2766] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:40.301816 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:40.368624 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788b803-a215-484d-bc7a-4e73ed7a06e1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:40.374767 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:40.375442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:40.375712 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:40.375981 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] PATH_INFO: `/v3/users/a03ba404d50f455db148c05bca9d5934` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:40.377168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a03ba404d50f455db148c05bca9d5934)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:40.379163 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:40.381589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:40.429805 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a03ba404d50f455db148c05bca9d5934'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:40.430628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a03ba404d50f455db148c05bca9d5934'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:40.437647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a03ba404d50f455db148c05bca9d5934'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:40.447777 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a03ba404d50f455db148c05bca9d5934'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:40.448613 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a03ba404d50f455db148c05bca9d5934'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:40.468670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1370/2767] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:40 2022] DELETE /identity/v3/users/a03ba404d50f455db148c05bca9d5934 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:40.473851 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:40.474364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:40.474474 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:40.495971 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:40.497178 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:40.558070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1398/2768] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:40.568330 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:40.569971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:40.636984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e789580-e7fe-4f06-99b1-8deb88e6a039 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:40.640756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:40.641370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:40.641679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:40.641948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:40.651695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:40.653165 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede4519-8a41-4f09-968f-b886f425728f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:40.662566 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1371/2769] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:40.948778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:40.950660 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:40.964238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501be28-8d01-4f18-aec1-45eabbb0f902 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:40.970477 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:40.971138 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:40.971548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:40.971822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] PATH_INFO: `/v3/projects/27df862560af4235a332f6145bbfc00c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:40.973071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27df862560af4235a332f6145bbfc00c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:40.975037 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.022954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27df862560af4235a332f6145bbfc00c', 'request_id': 'req-80d5642b-9399-4069-b261-b7883317411b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:41.032012 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27df862560af4235a332f6145bbfc00c', 'request_id': 'req-80d5642b-9399-4069-b261-b7883317411b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:41.037774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27df862560af4235a332f6145bbfc00c', 'request_id': 'req-80d5642b-9399-4069-b261-b7883317411b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:41.049091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] The token cache is being invalidate because project 27df862560af4235a332f6145bbfc00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:41.049374 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:41.050142 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-80d5642b-9399-4069-b261-b7883317411b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:41.051784 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1399/2770] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:40 2022] DELETE /identity/v3/projects/27df862560af4235a332f6145bbfc00c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:41.057069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.057352 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.057710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.078203 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:41.079326 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:41.142040 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1372/2771] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:41.150313 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb18-0263-4139-a05c-601221850281 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.150589 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb18-0263-4139-a05c-601221850281 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.150856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb18-0263-4139-a05c-601221850281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.172000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-739ecb18-0263-4139-a05c-601221850281 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:41.173220 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-739ecb18-0263-4139-a05c-601221850281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:41.182602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1400/2772] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:41.187206 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.197516 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9d9ef-d44f-4173-a304-7b2622d442b7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.203327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.203932 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.204179 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.204447 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.205149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:54:41.205455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.207117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.219402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1373/2773] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:54:41.223800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ecb18-0263-4139-a05c-601221850281 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.233337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ecb18-0263-4139-a05c-601221850281 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.238342 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.238945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.239219 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.239483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.240153 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.242072 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.271741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1401/2774] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:41 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:54:41.276060 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.286746 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f2bc8-2634-4459-806d-b8fae4a4e3de admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.295217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:41.347472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.348134 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.348232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.348517 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.349185 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.351180 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.375636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1374/2775] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:41 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:41.379885 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.386743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:41.395343 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29961e56-f2cb-40c0-b45e-b18380a57c36 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.404362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.405238 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.405605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.405945 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.406957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.409422 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.421622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1402/2776] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:41.425694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.436389 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a8002-b424-4e42-956b-c965056a7ae3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.442325 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.442931 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.443217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.443488 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.444192 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.445843 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d973c-a1f4-4f2e-a356-1f66fef05b4a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.456915 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1375/2777] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:41.461244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.472280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1af8ea-3e40-48f5-804e-9dcb02616985 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.478264 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.478967 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.479246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.479516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] PATH_INFO: `/v3/projects/649bab48321e4b06af511ec45cad5abe/users/029eb16091774e94bfc8ea2c8994b90b/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.498883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649bab48321e4b06af511ec45cad5abe, user_id=029eb16091774e94bfc8ea2c8994b90b, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.501604 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.518527 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1403/2778] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:41 2022] PUT /identity/v3/projects/649bab48321e4b06af511ec45cad5abe/users/029eb16091774e94bfc8ea2c8994b90b/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:54:41.586387 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.586774 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.586976 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.624803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] MFA Rules not processed for user `029eb16091774e94bfc8ea2c8994b90b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:41.625990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:41.697419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1376/2779] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:41.710576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:41.712869 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:41.790741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a70190-9a7c-42e2-b0c5-37b673597be1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:41.796852 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:41.797466 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:41.797743 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:41.798026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:41.807028 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:41.808609 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8e6d1-2041-4bb8-b0e9-a359b9392a14 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:41.818663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1404/2780] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:42.034656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:42.103943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da3ee-42aa-4c45-80ed-d03879800bc3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:42.110243 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:42.110995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.111268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.111536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] PATH_INFO: `/v3/users/029eb16091774e94bfc8ea2c8994b90b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.112765 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=029eb16091774e94bfc8ea2c8994b90b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:42.114815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:42.136979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:42.168943 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029eb16091774e94bfc8ea2c8994b90b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.169631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029eb16091774e94bfc8ea2c8994b90b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.177092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029eb16091774e94bfc8ea2c8994b90b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.185803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029eb16091774e94bfc8ea2c8994b90b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.186628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029eb16091774e94bfc8ea2c8994b90b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.204435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1377/2781] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:42 2022] DELETE /identity/v3/users/029eb16091774e94bfc8ea2c8994b90b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:42.209200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.209491 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.209800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.230299 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:42.231514 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:42.294899 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1405/2782] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:42.306763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:42.308386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:42.384694 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1dc39-8d30-4ee9-bb41-e8a5c8cca5e3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:42.388925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:42.389510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.389824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.390095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.399918 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:42.401364 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873780bf-d048-48f6-a1d0-d60170ae0f98 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:42.411059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1378/2783] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:42.692716 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:42.694276 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:42.706575 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d69442-9a35-40e1-890e-f9ac9696bc20 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:42.711450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:42.712439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.712916 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.713248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] PATH_INFO: `/v3/projects/649bab48321e4b06af511ec45cad5abe` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.714520 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=649bab48321e4b06af511ec45cad5abe)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:42.716445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:42.763210 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:42.764443 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649bab48321e4b06af511ec45cad5abe', 'request_id': 'req-16cbeda7-eca2-4a32-b111-6d78dcd04dec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.773890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649bab48321e4b06af511ec45cad5abe', 'request_id': 'req-16cbeda7-eca2-4a32-b111-6d78dcd04dec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.779656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649bab48321e4b06af511ec45cad5abe', 'request_id': 'req-16cbeda7-eca2-4a32-b111-6d78dcd04dec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.790731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] The token cache is being invalidate because project 649bab48321e4b06af511ec45cad5abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:42.791015 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.791695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16cbeda7-eca2-4a32-b111-6d78dcd04dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:42.793246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1406/2784] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:42 2022] DELETE /identity/v3/projects/649bab48321e4b06af511ec45cad5abe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:42.798471 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.798753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.799021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.818925 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:42.820127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:42.888658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1379/2785] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:42.900918 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.901256 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.901583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.927112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:42.928327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:42.938305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1407/2786] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:42.942874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:42.953392 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcf2ae-30a4-49a0-8b4d-ea34b57aad29 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:42.959123 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:42.959738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.960017 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.960289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.960994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:54:42.961300 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:42.963016 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:42.975485 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1380/2787] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:54:42.980070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:42.989871 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de29e5-1d31-4f32-a40c-fa140007d70a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:42.996471 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:42.997078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:42.997354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:42.997667 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:42.998328 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.000245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.034337 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1408/2788] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:42 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:43.040927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.057666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d5f17-da3d-45b4-836a-390ed2052275 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.090610 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:43.133576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.134241 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.134519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.134789 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.135454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.137385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.163289 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1381/2789] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:43 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:43.168475 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.189721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac329f-8e9c-48d0-8476-eb54dff508d1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.199324 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.200260 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.200642 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.201021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.202227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.204224 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.216122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1409/2790] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:43.223087 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.239636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793138-d349-4282-a6d5-caa39bc40075 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.246668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.247279 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.247556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.247819 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.248542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.250193 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edac34-bedf-43b0-afbb-d3fbef646186 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.260900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1382/2791] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:43.265896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.279622 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe55-c5fe-409d-93b9-b7ccff4d63d2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.285542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.286240 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.286526 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.286800 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] PATH_INFO: `/v3/projects/eb7e07bd54454dc5ab92870f06b62ae4/users/cb984ad0b6d540668f15c697ec279625/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.302882 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7e07bd54454dc5ab92870f06b62ae4, user_id=cb984ad0b6d540668f15c697ec279625, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.305549 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.321970 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1410/2792] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:43 2022] PUT /identity/v3/projects/eb7e07bd54454dc5ab92870f06b62ae4/users/cb984ad0b6d540668f15c697ec279625/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:43.332508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.332796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.333067 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.363996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] MFA Rules not processed for user `cb984ad0b6d540668f15c697ec279625`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:43.365225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:43.438211 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1383/2793] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:43.442856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.507618 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705128e-2b81-4fe3-972c-efdd6a2675cc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.512962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.513593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.513875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.514145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.514807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.516697 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.547614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1411/2794] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:43 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:43.552005 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.563038 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43823dd-d74e-4d5f-aa96-be8026e03031 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.624254 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.624910 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.625187 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.625455 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.626145 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.628128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.650555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1384/2795] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:43 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:43.655204 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.669081 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4116d-a780-487d-82d9-b3237a0ae729 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.674766 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.675385 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.675665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.675936 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.676638 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.678332 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.689698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1412/2796] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:43.693995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.704557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff321c3b-4f06-4489-81dd-cfdb82451f45 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.709977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.710669 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.710947 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.711220 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] PATH_INFO: `/v3/projects/281034bf5170440bbe5c3afc3c04b17f/users/cd84b6e7f7334f79975c63b5d03b259b/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.727994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=281034bf5170440bbe5c3afc3c04b17f, user_id=cd84b6e7f7334f79975c63b5d03b259b, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.730690 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.747513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1385/2797] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:43 2022] PUT /identity/v3/projects/281034bf5170440bbe5c3afc3c04b17f/users/cd84b6e7f7334f79975c63b5d03b259b/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:43.753451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.769777 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a6f7d-e2e4-4df5-80cd-39160e65c573 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.817155 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:43.832254 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.832879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.833156 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.833424 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.834151 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.835739 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d94812-bbe4-4b2b-b614-1ee5ac0e947a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.846176 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1413/2798] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:43.850729 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:43.859738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:43.866424 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751516-e1da-4627-95a7-8986531954eb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:43.871573 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:43.872368 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.872647 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.872938 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] PATH_INFO: `/v3/projects/281034bf5170440bbe5c3afc3c04b17f/users/cd84b6e7f7334f79975c63b5d03b259b/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.874959 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281034bf5170440bbe5c3afc3c04b17f, user_id=cd84b6e7f7334f79975c63b5d03b259b, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:43.877670 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:43.888835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1386/2799] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:43 2022] PUT /identity/v3/projects/281034bf5170440bbe5c3afc3c04b17f/users/cd84b6e7f7334f79975c63b5d03b259b/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:54:43.904359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:43.904678 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:43.904933 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:43.935502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] MFA Rules not processed for user `cd84b6e7f7334f79975c63b5d03b259b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:43.936731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:44.009799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1414/2800] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:54:44.020980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:44.022877 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:44.082575 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136318e-972e-4133-b631-2c51e8e42de3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:44.086666 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:44.088507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:44.088797 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:44.089070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:44.099259 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:44.101783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f1e4-146b-4ecc-ac06-ca1c12aba268 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:44.117024 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1387/2801] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:44.555104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:44.627252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae958d-c9eb-48a3-a67e-0ad062bff7cf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:44.632301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:44.632947 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:44.633221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:44.633593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] PATH_INFO: `/v3/users/cb984ad0b6d540668f15c697ec279625` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:44.634941 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb984ad0b6d540668f15c697ec279625)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:44.636864 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:44.689478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb984ad0b6d540668f15c697ec279625'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:44.698835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb984ad0b6d540668f15c697ec279625'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:44.699420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb984ad0b6d540668f15c697ec279625'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:44.700641 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb984ad0b6d540668f15c697ec279625'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:44.712026 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:44.713232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb984ad0b6d540668f15c697ec279625'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:44.728305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1415/2802] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:44 2022] DELETE /identity/v3/users/cb984ad0b6d540668f15c697ec279625 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:44.732966 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:44.733472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:44.733472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:44.753342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:44.754626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:44.818580 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1388/2803] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:44.828587 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:44.830218 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:44.888672 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c63be-f906-476b-b634-a3c8799d1d1b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:44.894019 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:44.894893 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:44.895277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:44.895661 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:44.909047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:44.910789 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:44.920801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1416/2804] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:45.227047 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:45.228680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:45.241445 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5504e-cb0a-474b-b267-753c5e06fe15 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:45.246738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:45.247394 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:45.247677 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:45.247935 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] PATH_INFO: `/v3/projects/eb7e07bd54454dc5ab92870f06b62ae4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:45.249139 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7e07bd54454dc5ab92870f06b62ae4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:45.251090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:45.304358 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7e07bd54454dc5ab92870f06b62ae4', 'request_id': 'req-fea2fb98-0d32-4084-9a5c-5b38b9415a35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.315803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7e07bd54454dc5ab92870f06b62ae4', 'request_id': 'req-fea2fb98-0d32-4084-9a5c-5b38b9415a35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.321607 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7e07bd54454dc5ab92870f06b62ae4', 'request_id': 'req-fea2fb98-0d32-4084-9a5c-5b38b9415a35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.332537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] The token cache is being invalidate because project eb7e07bd54454dc5ab92870f06b62ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:45.332815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.333429 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.334975 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1389/2805] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:45 2022] DELETE /identity/v3/projects/eb7e07bd54454dc5ab92870f06b62ae4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:45.339479 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:45.350323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1fcc1-1a90-4f0e-bc24-276772fdd7ad service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:45.412118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:45.412762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:45.413041 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:45.413316 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] PATH_INFO: `/v3/users/cd84b6e7f7334f79975c63b5d03b259b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:45.414595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd84b6e7f7334f79975c63b5d03b259b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:45.416584 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:45.472500 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd84b6e7f7334f79975c63b5d03b259b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.481825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd84b6e7f7334f79975c63b5d03b259b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.482538 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd84b6e7f7334f79975c63b5d03b259b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.483216 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd84b6e7f7334f79975c63b5d03b259b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.498213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ead2718a-3c81-49bd-be77-7ca5d90bcde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd84b6e7f7334f79975c63b5d03b259b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.510849 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1417/2806] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:45 2022] DELETE /identity/v3/users/cd84b6e7f7334f79975c63b5d03b259b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:45.796600 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:45.798089 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:45.813048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2fb98-0d32-4084-9a5c-5b38b9415a35 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:45.826442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:45.877151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:45.877768 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:45.878048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:45.878316 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] PATH_INFO: `/v3/projects/281034bf5170440bbe5c3afc3c04b17f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:45.879480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=281034bf5170440bbe5c3afc3c04b17f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:45.881385 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:45.934757 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281034bf5170440bbe5c3afc3c04b17f', 'request_id': 'req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.943837 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281034bf5170440bbe5c3afc3c04b17f', 'request_id': 'req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.950415 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281034bf5170440bbe5c3afc3c04b17f', 'request_id': 'req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.962382 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] The token cache is being invalidate because project 281034bf5170440bbe5c3afc3c04b17f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:45.962638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.963462 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b4f425d9-58fc-4cf3-82d5-fa7c769ebb2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:45.965268 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1390/2807] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:45 2022] DELETE /identity/v3/projects/281034bf5170440bbe5c3afc3c04b17f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 22:54:45.972271 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:45.972797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:45.972797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:45.995031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:45.996234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:46.000974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:46.060531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1418/2808] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:54:46.072178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.072399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.072813 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.098864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:46.100081 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:46.108639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1391/2809] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 22:54:46.113175 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.123644 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dda73f-d49a-4e76-bf1d-0d4360cefaad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.128714 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.129258 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.129533 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.129809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.130539 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:54:46.130816 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.132437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.144566 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1419/2810] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:54:46.149821 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.163894 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e08789-d89f-42e3-9737-edf95a7ed4db None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.171052 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.171833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.172147 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.172480 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.173156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.175437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.213948 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1392/2811] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:46.218372 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.229112 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7eca6-3eae-4693-91ad-c5fa96f0fe2a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.297091 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.297909 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.298311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.298688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.299655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.302361 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.328420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1420/2812] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:46.333648 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.345599 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e9233-8b0b-4cdf-8a88-d3c21dd233ab admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.351477 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.352246 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.354264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.354264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.354264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.355142 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.367960 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1393/2813] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:46.372690 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.383452 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1b852-16e2-423d-835a-0042d5eee8fd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.388862 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.389493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.389803 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.390067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.390759 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.392368 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e0a77-5a8d-40aa-a8f0-9163dbd5cd23 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.404288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1421/2814] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:46.408986 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.420051 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb6d7b-ef9e-41e1-8310-ffef2a680ba5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.425603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.426320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.426591 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.426862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] PATH_INFO: `/v3/projects/d96935a32a2b47ddafa52950842207a5/users/7929276a633a4933a9ae3c975d236fa6/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.443128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d96935a32a2b47ddafa52950842207a5, user_id=7929276a633a4933a9ae3c975d236fa6, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.445873 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.460990 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1394/2815] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:46 2022] PUT /identity/v3/projects/d96935a32a2b47ddafa52950842207a5/users/7929276a633a4933a9ae3c975d236fa6/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:46.474152 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.474405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.474695 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.504451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] MFA Rules not processed for user `7929276a633a4933a9ae3c975d236fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:46.505734 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:46.572065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1422/2816] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:46.576833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.622501 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:46.643054 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86558f-77c3-46c9-b118-4d29e19a375b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.648328 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.648955 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.649237 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.649508 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.650231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.652161 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.682417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1395/2817] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:46.686721 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.693513 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:46.698554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caec4e6-4eb5-4984-8444-40ecbfdf311c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.756923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.757578 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.757875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.758179 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.758879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.760854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.783386 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1423/2818] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:46 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:46.787731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.798944 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4093-f283-428b-98a7-5cbf3a50ec53 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.804103 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.804720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.804996 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.805266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.806128 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.807730 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.818631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1396/2819] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:46.823288 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.834230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225bfc-3a78-43f8-8b1f-7e1cfb914038 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.840418 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.841213 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.841499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.841808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] PATH_INFO: `/v3/projects/73c7c01f3d8c4b218b192639106397cc/users/2bf1b125adff48c1932b736ba7557234/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.857907 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c7c01f3d8c4b218b192639106397cc, user_id=2bf1b125adff48c1932b736ba7557234, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.860517 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.874476 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1424/2820] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:46 2022] PUT /identity/v3/projects/73c7c01f3d8c4b218b192639106397cc/users/2bf1b125adff48c1932b736ba7557234/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:46.878620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.889513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd1f24-a0ee-460a-83a1-6badccf066e6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.953971 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.954612 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.954888 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.955152 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.955847 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.957464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696709f-f9ac-4bd2-8526-7ad74113b12c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:46.968263 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1397/2821] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 22:54:46.973021 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:46.984431 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af702a-f9e4-4bff-ab39-43d1cf45f32d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:46.990223 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:46.990883 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:46.991158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:46.991426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] PATH_INFO: `/v3/projects/73c7c01f3d8c4b218b192639106397cc/users/2bf1b125adff48c1932b736ba7557234/roles/c512669c168542d587f922137072be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:46.993651 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c7c01f3d8c4b218b192639106397cc, user_id=2bf1b125adff48c1932b736ba7557234, role_id=c512669c168542d587f922137072be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:46.996326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:47.008312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1425/2822] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:46 2022] PUT /identity/v3/projects/73c7c01f3d8c4b218b192639106397cc/users/2bf1b125adff48c1932b736ba7557234/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:47.019262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:47.019543 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:47.019895 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:47.048247 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] MFA Rules not processed for user `2bf1b125adff48c1932b736ba7557234`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:47.049454 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:47.129714 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1398/2823] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:47.141953 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:47.143814 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:47.208707 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44b94-3cf9-4ec4-8380-06c073ab63fb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:47.212614 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:47.213230 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:47.213504 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:47.213809 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:47.224201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:47.225668 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82dbf0-610e-4a7c-8feb-94f6d2dd6990 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:47.235464 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1426/2824] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:47.702869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:47.747064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:47.770679 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75a9a-c668-4fa3-aa7d-65e39ef349c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:47.776056 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:47.776751 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:47.777070 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:47.777386 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] PATH_INFO: `/v3/users/7929276a633a4933a9ae3c975d236fa6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:47.778689 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7929276a633a4933a9ae3c975d236fa6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:47.780685 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:47.830929 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7929276a633a4933a9ae3c975d236fa6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:47.831597 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7929276a633a4933a9ae3c975d236fa6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:47.838653 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7929276a633a4933a9ae3c975d236fa6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:47.847718 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7929276a633a4933a9ae3c975d236fa6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:47.848363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7929276a633a4933a9ae3c975d236fa6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:47.866949 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1399/2825] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:47 2022] DELETE /identity/v3/users/7929276a633a4933a9ae3c975d236fa6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:47.872487 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:47.872767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:47.873038 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:47.892423 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:47.893583 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:47.953040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:47.963516 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1427/2826] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:47.973946 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:47.975458 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:48.051510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b7a-6561-4dee-bf30-33b5187f8210 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:48.055722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:48.056347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:48.056622 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:48.056891 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:48.066893 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:48.068344 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:48.078908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1400/2827] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:48.386683 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:48.388318 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:48.405655 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348c9fd-c12e-49bc-85df-90e1fa1069f8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:48.411036 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:48.411698 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:48.411991 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:48.412323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] PATH_INFO: `/v3/projects/d96935a32a2b47ddafa52950842207a5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:48.413497 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d96935a32a2b47ddafa52950842207a5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:48.415451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:48.467167 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd96935a32a2b47ddafa52950842207a5', 'request_id': 'req-519b1704-564b-483b-b36c-7fcaaaf76969'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.479166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd96935a32a2b47ddafa52950842207a5', 'request_id': 'req-519b1704-564b-483b-b36c-7fcaaaf76969'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.484856 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd96935a32a2b47ddafa52950842207a5', 'request_id': 'req-519b1704-564b-483b-b36c-7fcaaaf76969'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.496694 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] The token cache is being invalidate because project d96935a32a2b47ddafa52950842207a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:48.497029 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.497706 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.499327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1428/2828] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:48 2022] DELETE /identity/v3/projects/d96935a32a2b47ddafa52950842207a5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:48.503556 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:48.513890 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b938e1-1b7b-478d-85e7-e8e06b9a0da2 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:48.573697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:48.574258 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:48.574536 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:48.574803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] PATH_INFO: `/v3/users/2bf1b125adff48c1932b736ba7557234` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:48.576023 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf1b125adff48c1932b736ba7557234)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:48.577984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:48.579644 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:48.632305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf1b125adff48c1932b736ba7557234'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.633225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf1b125adff48c1932b736ba7557234'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.641064 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf1b125adff48c1932b736ba7557234'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.649951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf1b125adff48c1932b736ba7557234'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.650399 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ca71b3c-85e8-4396-a4cc-89c5c03913e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf1b125adff48c1932b736ba7557234'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:48.668519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1401/2829] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:54:48 2022] DELETE /identity/v3/users/2bf1b125adff48c1932b736ba7557234 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:48.967774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:48.969585 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:48.983776 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b1704-564b-483b-b36c-7fcaaaf76969 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.045493 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.046227 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.046502 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.046770 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] PATH_INFO: `/v3/projects/73c7c01f3d8c4b218b192639106397cc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.047954 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c7c01f3d8c4b218b192639106397cc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.049915 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.098351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c7c01f3d8c4b218b192639106397cc', 'request_id': 'req-3e58e6bf-be28-4ff1-8db2-88593b6e1337'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:49.107607 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c7c01f3d8c4b218b192639106397cc', 'request_id': 'req-3e58e6bf-be28-4ff1-8db2-88593b6e1337'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:49.113345 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c7c01f3d8c4b218b192639106397cc', 'request_id': 'req-3e58e6bf-be28-4ff1-8db2-88593b6e1337'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:49.124568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] The token cache is being invalidate because project 73c7c01f3d8c4b218b192639106397cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:54:49.124873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:49.125612 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3e58e6bf-be28-4ff1-8db2-88593b6e1337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:54:49.128023 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1429/2830] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:54:48 2022] DELETE /identity/v3/projects/73c7c01f3d8c4b218b192639106397cc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:49.134658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.135173 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.135275 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.158456 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:49.159678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:49.221869 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1402/2831] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:49.230251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.230531 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.230797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.251729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:49.252930 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:49.261994 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1430/2832] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:49.267833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.279994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625779b9-f4f6-44aa-9d2a-e710d3844bbb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.287049 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.287862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.288197 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.288529 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.289494 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:54:49.289927 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.292428 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.309907 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1403/2833] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:54:49.314446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.324505 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0552-f309-4563-afbf-c1376c7f5f73 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.330229 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.330840 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.331114 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.331382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.332071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.334031 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.338957 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:49.365427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1431/2834] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:49.369871 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.379937 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8fab9-f94a-4508-84c4-ebed522c7f87 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.436884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.437528 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.437820 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.438096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.438753 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.440682 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.451676 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:49.466384 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1404/2835] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:49.471157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.482074 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972368b-14dc-4ec1-8468-f54c813b8470 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.487845 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.488483 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.488788 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.489062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.489804 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.491392 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.502013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1432/2836] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:49.507762 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.518759 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb4d3-db8f-477b-8638-e7f283f2d331 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.524662 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.525217 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.525495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.525796 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.526495 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.528112 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990f021-9e95-44cb-85f0-93b8bec6a11e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.539218 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1405/2837] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:49.544786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.555627 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1856b98-99e0-4e63-892a-361642395606 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.561593 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.562277 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.562560 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.562830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] PATH_INFO: `/v3/projects/6de70703f2234fa58f21bec1d2ebed1e/users/e9c55f77d85b4ebeae2c4a429bd233b8/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.578364 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de70703f2234fa58f21bec1d2ebed1e, user_id=e9c55f77d85b4ebeae2c4a429bd233b8, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.581090 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.596973 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1433/2838] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:49 2022] PUT /identity/v3/projects/6de70703f2234fa58f21bec1d2ebed1e/users/e9c55f77d85b4ebeae2c4a429bd233b8/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:49.608616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.608900 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.609156 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.638096 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] MFA Rules not processed for user `e9c55f77d85b4ebeae2c4a429bd233b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:49.639226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:49.703110 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1406/2839] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:49.707740 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.771326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9308-3c64-4950-acba-e2ccc50c51e5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.776550 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.777161 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.777439 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.777742 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.778409 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.780339 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.811237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1434/2840] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:49.815484 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.826437 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb7178-6180-45a1-823c-deea72d20b9e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.882510 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.883124 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.883400 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.883652 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.884310 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.886273 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.910370 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1407/2841] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:54:49 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:54:49.914636 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.925675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8168b-2055-4798-aa4c-777007f29197 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.932249 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.932896 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.933173 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.933444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.934180 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.935778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:49.946783 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1435/2842] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:49.951167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:49.961423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ae85-5b4b-4455-8390-ee251fb9f447 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:49.966611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:49.967266 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:49.967539 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:49.967841 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] PATH_INFO: `/v3/projects/a37f74cc8a494e708a15b389abe057f7/users/80338fdd527b4bcd895a3814b055b707/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:49.983391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f74cc8a494e708a15b389abe057f7, user_id=80338fdd527b4bcd895a3814b055b707, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:49.986090 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:50.001151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1408/2843] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:49 2022] PUT /identity/v3/projects/a37f74cc8a494e708a15b389abe057f7/users/80338fdd527b4bcd895a3814b055b707/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:50.005116 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:50.016056 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efdf10-078c-48d1-9874-fb503a14d4d9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:50.074741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:50.075403 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:50.075665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:50.075908 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:50.076657 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:50.078311 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201fe4-df39-439d-9447-7f35a11523ea admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:50.088648 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1436/2844] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:54:50.093122 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:50.104823 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32ef6e-3324-402d-b991-c0a66924d898 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:50.110864 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:50.111538 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:50.111840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:50.112113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] PATH_INFO: `/v3/projects/a37f74cc8a494e708a15b389abe057f7/users/80338fdd527b4bcd895a3814b055b707/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:50.114347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f74cc8a494e708a15b389abe057f7, user_id=80338fdd527b4bcd895a3814b055b707, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:50.117059 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:50.128557 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1409/2845] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:54:50 2022] PUT /identity/v3/projects/a37f74cc8a494e708a15b389abe057f7/users/80338fdd527b4bcd895a3814b055b707/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:54:50.139137 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:50.139426 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:50.139703 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:50.168524 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] MFA Rules not processed for user `80338fdd527b4bcd895a3814b055b707`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:54:50.169686 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:50.178675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:50.246482 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1437/2846] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:54:50.258322 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:50.260668 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:50.295094 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:54:50.324621 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce078-8034-4de9-932b-0e9af4eb15f5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:50.329148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:50.329809 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:50.330079 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:50.330351 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:50.397256 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:50.398754 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc61bd-e34e-45d8-af49-6a4df5e93b8a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:50.408225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1410/2847] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:54:56.102598 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:54:56.104608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:54:56.118231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460db192-3533-4959-b6ef-80eaf49e6ff3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:54:56.122832 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a57435ddd44fe79517001c2d91eec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:54:56.123451 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:54:56.123731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:54:56.124000 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:54:56.133879 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:54:56.135294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:54:56.145428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1438/2848] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:54:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:00.404232 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=36 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:06.140787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=31 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:12.258855 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:12.259489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:12.259489 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:12.282360 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] MFA Rules not processed for user `4bba292314fd415f94fb537237d9d898`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:12.284010 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:12.353075 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1411/2849] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:12.715150 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:12.716830 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:12.783630 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58843b-e439-46ea-b870-d327e01c02b7 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:12.788070 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:12.788723 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:12.788999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:12.789265 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:12.791457 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:12.802762 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:12.804232 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:12.813817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1439/2850] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:13.333006 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:13.334720 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:13.419776 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fef628-3f0b-4b26-9547-f2d43ec66451 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:13.425169 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:13.425858 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:13.426144 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:13.426419 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:13.428262 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:13.440710 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:13.442391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36edfb-0e8b-4158-a0e9-39542ee5913f service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:13.453397 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1412/2851] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:20.480571 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:20.561807 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ed713-80e2-4beb-847a-e8e9a005a4b2 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:20.566713 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:20.567430 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:20.567709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:20.567997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] PATH_INFO: `/v3/users/e9c55f77d85b4ebeae2c4a429bd233b8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:20.569274 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c55f77d85b4ebeae2c4a429bd233b8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:20.571283 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:20.629753 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c55f77d85b4ebeae2c4a429bd233b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:20.639810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c55f77d85b4ebeae2c4a429bd233b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:20.640450 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c55f77d85b4ebeae2c4a429bd233b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:20.641065 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c55f77d85b4ebeae2c4a429bd233b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:20.654208 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c55f77d85b4ebeae2c4a429bd233b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:20.666323 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1440/2852] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:55:20 2022] DELETE /identity/v3/users/e9c55f77d85b4ebeae2c4a429bd233b8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:20.671329 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:20.672007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:20.672007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:20.698654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:20.699968 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:20.761306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1413/2853] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:20.772719 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:20.774388 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:20.803251 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:20.836890 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d089cbb-446f-4c5a-876e-22aac060524c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:20.841077 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:20.841751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:20.842024 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:20.842294 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:20.853460 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:20.854940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:20.872118 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1441/2854] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:55:21.160879 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:21.162564 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:21.177166 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be96e4-4f95-4f5c-9f43-fe7ad920beac None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:21.182810 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:21.183639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:21.183920 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:21.184195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] PATH_INFO: `/v3/projects/6de70703f2234fa58f21bec1d2ebed1e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:21.185440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de70703f2234fa58f21bec1d2ebed1e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:21.187638 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:21.228312 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:21.244141 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de70703f2234fa58f21bec1d2ebed1e', 'request_id': 'req-09a6df2f-3c5c-493c-a795-885a01c44b74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.253031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de70703f2234fa58f21bec1d2ebed1e', 'request_id': 'req-09a6df2f-3c5c-493c-a795-885a01c44b74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.259226 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de70703f2234fa58f21bec1d2ebed1e', 'request_id': 'req-09a6df2f-3c5c-493c-a795-885a01c44b74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.270691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] The token cache is being invalidate because project 6de70703f2234fa58f21bec1d2ebed1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:55:21.270980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.271639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.273231 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1414/2855] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:55:21 2022] DELETE /identity/v3/projects/6de70703f2234fa58f21bec1d2ebed1e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:21.277646 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:21.288157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044ab1d-5b31-4717-bc5b-a2591a07ed0d service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:21.360289 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:21.360876 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:21.361154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:21.361427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] PATH_INFO: `/v3/users/80338fdd527b4bcd895a3814b055b707` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:21.362763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80338fdd527b4bcd895a3814b055b707)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:21.364733 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:21.421741 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80338fdd527b4bcd895a3814b055b707'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.430865 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80338fdd527b4bcd895a3814b055b707'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.431488 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80338fdd527b4bcd895a3814b055b707'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.432233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80338fdd527b4bcd895a3814b055b707'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.445183 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d37cd443-f00d-486f-ad7e-2feb7266c6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80338fdd527b4bcd895a3814b055b707'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.459062 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1442/2856] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:55:21 2022] DELETE /identity/v3/users/80338fdd527b4bcd895a3814b055b707 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:21.766253 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:21.768099 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:21.783570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6df2f-3c5c-493c-a795-885a01c44b74 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:21.849209 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:21.849924 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:21.850149 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:21.850441 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] PATH_INFO: `/v3/projects/a37f74cc8a494e708a15b389abe057f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:21.851589 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37f74cc8a494e708a15b389abe057f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:21.853507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:21.903007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37f74cc8a494e708a15b389abe057f7', 'request_id': 'req-73b6d7fe-4078-4518-aa0c-44c31b75ec74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.912565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37f74cc8a494e708a15b389abe057f7', 'request_id': 'req-73b6d7fe-4078-4518-aa0c-44c31b75ec74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.918980 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37f74cc8a494e708a15b389abe057f7', 'request_id': 'req-73b6d7fe-4078-4518-aa0c-44c31b75ec74'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.931388 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] The token cache is being invalidate because project a37f74cc8a494e708a15b389abe057f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:55:21.931671 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.932381 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-73b6d7fe-4078-4518-aa0c-44c31b75ec74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:55:21.933995 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1415/2857] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:55:21 2022] DELETE /identity/v3/projects/a37f74cc8a494e708a15b389abe057f7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:21.940557 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:21.941119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:21.941119 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:21.961304 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:21.962548 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:22.006108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:22.032002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1443/2858] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:22.041542 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.041849 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.042117 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.063702 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:22.064936 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:22.073732 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1416/2859] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:22.078354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.089468 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adb330-af49-4c8f-b004-4ce67401ed0d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.096166 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.096801 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.097078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.097346 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.098104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 22:55:22.098444 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.100068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.113378 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1444/2860] 10.222.0.47 () {58 vars in 1153 bytes} [Mon Aug 1 22:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 22:55:22.118131 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.128348 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e582da3-0c3b-4441-979b-e6bb1063f6c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.134442 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.135113 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.135391 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.135658 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.136342 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.138417 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.172464 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1417/2861] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:55:22 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:55:22.176932 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.187799 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8c24c-e4fb-4f9f-9947-a8f26455c2b8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.254047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.254687 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.254964 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.255234 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.255942 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.257935 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.283445 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1445/2862] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:55:22 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:55:22.288162 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.299632 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c333cf-fcb0-4c1a-a1c5-cd403486a944 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.304738 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.305376 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.305695 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.305972 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.306691 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.308320 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.319569 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1418/2863] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:55:22.323959 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.335061 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde84bd7-36c5-4964-9573-6df98bc889f6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.340243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.340878 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.341154 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.341427 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.342170 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.343785 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93568c-032b-4cf7-bd77-633ad9151b2a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.355931 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1446/2864] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:55:22.360476 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.371950 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f43178-6d5c-4c05-b1fc-8f0cf16b20a1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.377250 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.377979 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.378260 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.378537 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] PATH_INFO: `/v3/projects/243a702bf7464281a76960b63e849b76/users/5ff80fefdd364fc1b0a9a4c54c1e1a30/roles/d9a82ca5f4b244f49e8a8ba0d7999547` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.382127 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:22.396840 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243a702bf7464281a76960b63e849b76, user_id=5ff80fefdd364fc1b0a9a4c54c1e1a30, role_id=d9a82ca5f4b244f49e8a8ba0d7999547)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.399581 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.413626 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1419/2865] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:55:22 2022] PUT /identity/v3/projects/243a702bf7464281a76960b63e849b76/users/5ff80fefdd364fc1b0a9a4c54c1e1a30/roles/d9a82ca5f4b244f49e8a8ba0d7999547 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:22.419002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.419243 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.419472 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.440189 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] MFA Rules not processed for user `47de1e2ad35749d7a331fc838348d562`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:22.441340 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:22.510158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1447/2866] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:22.520544 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:22.522650 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:22.588766 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297ca8-36b5-41ae-99d5-bc125ff7d3d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:22.593214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:22.593892 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:22.594533 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:22.594803 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:22.605592 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:22.607629 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e258-74be-4591-b0bf-d743b775ec44 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:22.619244 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1420/2867] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:27.278621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:27.280866 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:27.394353 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:27.434123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89fb3a-a0ab-4803-9719-dbee025f6495 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:27.442045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:27.442822 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.443135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.443528 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.445966 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:27.460670 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:27.462440 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:27.477149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1448/2868] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:27.568336 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.568862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.568862 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.614625 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] MFA Rules not processed for user `5ff80fefdd364fc1b0a9a4c54c1e1a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:27.616225 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:27.723825 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1421/2869] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:27.728680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:27.740629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438388a-c75d-4287-b2a3-288f1c4dee02 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:27.746320 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:27.747629 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.748314 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.748810 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.749751 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:27.751875 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:27.784512 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1449/2870] 10.222.0.47 () {60 vars in 1153 bytes} [Mon Aug 1 22:55:27 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 22:55:27.788740 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:27.800654 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e658a-f02c-4f83-be05-8a0f9ba82448 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:27.865155 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:27.865815 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.866092 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.866343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.867033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:27.869393 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:27.898413 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1422/2871] 10.222.0.47 () {60 vars in 1144 bytes} [Mon Aug 1 22:55:27 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 22:55:27.904428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:27.922831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f67d2-7877-4e82-8b14-cedeea059152 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:27.929326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:27.930064 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.930327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.930595 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.931319 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:27.932987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:27.944796 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1450/2872] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:55:27.949194 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:27.960025 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ea64-fa0b-4c9f-a477-6328b758de62 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:27.964874 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:27.965699 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:27.965941 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:27.966210 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] PATH_INFO: `/v3/projects/613715b645fb404eb228e746ca3fd077/users/2bd587c1c2b94aee9ced936892c3c059/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:27.968940 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:27.986829 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613715b645fb404eb228e746ca3fd077, user_id=2bd587c1c2b94aee9ced936892c3c059, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:27.989534 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:28.009157 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1423/2873] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:55:27 2022] PUT /identity/v3/projects/613715b645fb404eb228e746ca3fd077/users/2bd587c1c2b94aee9ced936892c3c059/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:28.013621 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:28.024834 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733114f1-f43e-4bb0-8fbb-f31afae95be2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:28.100184 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:28.100720 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:28.101001 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:28.101270 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:28.102108 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:28.103744 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:28.115278 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1451/2874] 10.222.0.47 () {58 vars in 1122 bytes} [Mon Aug 1 22:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 22:55:28.120012 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:28.131865 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d6dda-e9a7-49ce-8965-dcaf3d6364a6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:28.138513 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:28.139248 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:28.139532 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:28.139812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] PATH_INFO: `/v3/projects/613715b645fb404eb228e746ca3fd077/users/2bd587c1c2b94aee9ced936892c3c059/roles/c512669c168542d587f922137072be5f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:28.142034 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=613715b645fb404eb228e746ca3fd077, user_id=2bd587c1c2b94aee9ced936892c3c059, role_id=c512669c168542d587f922137072be5f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:28.144857 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806ed2-4321-4729-99a2-5a746a0515ac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:28.157631 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1424/2875] 10.222.0.47 () {60 vars in 1483 bytes} [Mon Aug 1 22:55:28 2022] PUT /identity/v3/projects/613715b645fb404eb228e746ca3fd077/users/2bd587c1c2b94aee9ced936892c3c059/roles/c512669c168542d587f922137072be5f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:55:33.537099 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:33.540349 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:33.634940 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10402bd-b8ce-412d-8a32-09fd027ae083 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:33.686007 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:33.732047 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:33.732611 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:33.732969 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:33.733246 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:33.735454 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:33.753067 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:33.754763 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:33.768499 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1452/2876] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:34.323812 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1583f1-f76d-4cce-89f9-b6d069bde18e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:34.324756 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1583f1-f76d-4cce-89f9-b6d069bde18e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:34.325236 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1583f1-f76d-4cce-89f9-b6d069bde18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:34.371603 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8a1583f1-f76d-4cce-89f9-b6d069bde18e None None] MFA Rules not processed for user `2bd587c1c2b94aee9ced936892c3c059`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:34.373390 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8a1583f1-f76d-4cce-89f9-b6d069bde18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:34.458068 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1425/2877] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:34.470887 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:34.472306 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:34.566554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433f0a-d3e0-4288-8625-90bbc9d6ac81 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:34.571159 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f05345f84a448f3842f4e91d9f0ac67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:34.571997 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:34.572354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:34.572831 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:34.645654 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:34.647221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:34.657104 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1453/2878] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:34.797611 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45e07-4d19-45c3-bbeb-c80b432611e6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:34.797901 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45e07-4d19-45c3-bbeb-c80b432611e6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:34.798168 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd45e07-4d19-45c3-bbeb-c80b432611e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:34.863555 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ddd45e07-4d19-45c3-bbeb-c80b432611e6 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:34.864790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ddd45e07-4d19-45c3-bbeb-c80b432611e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:34.943304 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1426/2879] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:55:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:34.995934 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:34.997367 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:35.065045 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:35.098171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3728-ef40-41d4-9bff-7a15238619a8 service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:35.103387 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:35.104006 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:35.104279 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:35.104546 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:35.113057 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:35.114693 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:35.124131 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1454/2880] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:55:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:55:43.701097 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a46b-cf3b-49c2-a5ea-f23fc3c3bc45 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:43.701726 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a46b-cf3b-49c2-a5ea-f23fc3c3bc45 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:43.702158 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a46b-cf3b-49c2-a5ea-f23fc3c3bc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:43.725380 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c747a46b-cf3b-49c2-a5ea-f23fc3c3bc45 None None] MFA Rules not processed for user `5ff80fefdd364fc1b0a9a4c54c1e1a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:43.726636 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c747a46b-cf3b-49c2-a5ea-f23fc3c3bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:43.733783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1427/2881] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:55:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:44.020726 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:44.078351 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3b719-79c3-42aa-a4d2-9b2137fa97d6 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:44.082071 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:44.082684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:44.082961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:44.083231 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:44.085262 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:44.095519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:44.096939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:44.105010 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1455/2882] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:55:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:55:45.740781 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:45.741305 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:45.741409 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:45.762278 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:55:45.763507 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:55:45.771628 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1428/2883] 10.222.0.47 () {58 vars in 1009 bytes} [Mon Aug 1 22:55:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:55:45.784405 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:55:45.800501 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd62abe-b50f-4620-8d9c-564b280c2b9f service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:55:45.806053 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:55:45.806779 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:55:45.807102 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:55:45.807486 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:55:45.817120 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:55:45.818778 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:55:45.827558 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1456/2884] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:55:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:55:55.756977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:55:55.823853 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=47 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:56:12.889069 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:56:12.890905 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:56:12.904030 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64e87b-ab68-4329-a053-4ebf605d6e6f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:56:12.910002 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:56:12.910982 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:56:12.911264 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:56:12.911535 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:56:12.921884 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:56:12.923327 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6213-fe57-4734-88d3-61872d4bb863 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:56:12.933468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1429/2885] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:56:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:56:21.077123 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:56:21.078665 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:56:21.156688 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489fb2-3278-4869-aa4a-15695666b89a service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:56:21.162355 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:56:21.162757 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:56:21.163032 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:56:21.163301 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:56:21.165233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:56:21.178040 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:56:21.179602 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:56:21.190196 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1457/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:56:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:56:22.928911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:56:31.185579 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=32 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:57:23.207135 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892540-0db2-46c4-aaab-e4569c6a21de None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:23.207742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892540-0db2-46c4-aaab-e4569c6a21de None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:23.207742 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892540-0db2-46c4-aaab-e4569c6a21de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:23.229199 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-89892540-0db2-46c4-aaab-e4569c6a21de None None] MFA Rules not processed for user `4bba292314fd415f94fb537237d9d898`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:57:23.230574 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-89892540-0db2-46c4-aaab-e4569c6a21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:23.290635 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1430/2887] 10.222.0.47 () {58 vars in 958 bytes} [Mon Aug 1 22:57:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:57:33.283460 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=25 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:57:34.233188 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:34.278774 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497e7-7076-4506-8396-9d5182d39344 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:34.294171 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:34.295606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:34.295606 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:34.295826 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:34.301470 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:34.328478 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:34.330312 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:34.345858 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1458/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:57:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:57:41.946341 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:41.946863 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:41.947007 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:41.968562 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] MFA Rules not processed for user `feed14a81a6146338abb5ed7f7726e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 22:57:41.970306 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:41.978505 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1431/2889] 10.222.0.47 () {58 vars in 967 bytes} [Mon Aug 1 22:57:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 22:57:42.025699 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:42.038245 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d1a58-f622-4227-ac41-e37883682ec1 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:42.043999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '223d4641075e4b6489e3ee1e4fbdce36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:42.044617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:42.044891 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:42.045158 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:42.054247 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:42.056013 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:42.065149 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1459/2890] 10.222.0.47 () {60 vars in 1384 bytes} [Mon Aug 1 22:57:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 22:57:46.071548 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:46.075835 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:46.093048 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ee998-2d3a-4fe5-af70-5a5a7fd24f93 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:46.099509 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:46.100269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:46.100549 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:46.100822 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:46.102839 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:46.120512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:46.120512 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:46.132343 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1432/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:57:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:57:49.389221 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:49.404671 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607098d-8bc1-41b4-b7b0-d822ed4e0fba service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:49.410200 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:49.411326 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:49.411974 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:49.412605 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:49.414680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:49.429999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:49.431812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:49.453870 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1460/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:57:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:57:49.504229 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:49.589257 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0773e6-1416-4ad0-adce-36e3591cffb7 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:49.598233 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:49.598911 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:49.599435 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:49.599715 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] PATH_INFO: `/v3/users/5ff80fefdd364fc1b0a9a4c54c1e1a30` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:49.601027 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff80fefdd364fc1b0a9a4c54c1e1a30)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:49.603044 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:49.664697 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff80fefdd364fc1b0a9a4c54c1e1a30'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:49.666095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff80fefdd364fc1b0a9a4c54c1e1a30'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:49.675606 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff80fefdd364fc1b0a9a4c54c1e1a30'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:49.685678 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff80fefdd364fc1b0a9a4c54c1e1a30'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:49.686469 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff80fefdd364fc1b0a9a4c54c1e1a30'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:49.709614 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1433/2893] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:57:49 2022] DELETE /identity/v3/users/5ff80fefdd364fc1b0a9a4c54c1e1a30 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:57:49.912327 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:49.932034 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa9969-3ddd-42ef-84ed-56e7c6901740 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:50.015135 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:50.016317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:50.016317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:50.016507 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] PATH_INFO: `/v3/projects/243a702bf7464281a76960b63e849b76` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:50.017679 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243a702bf7464281a76960b63e849b76)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:50.019632 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:50.070305 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243a702bf7464281a76960b63e849b76', 'request_id': 'req-63b3690c-fe62-419e-b586-0bca7f1d5f94'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.078266 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243a702bf7464281a76960b63e849b76', 'request_id': 'req-63b3690c-fe62-419e-b586-0bca7f1d5f94'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.084362 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243a702bf7464281a76960b63e849b76', 'request_id': 'req-63b3690c-fe62-419e-b586-0bca7f1d5f94'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.099437 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] The token cache is being invalidate because project 243a702bf7464281a76960b63e849b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:57:50.099729 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.100576 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.102244 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1461/2894] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:57:49 2022] DELETE /identity/v3/projects/243a702bf7464281a76960b63e849b76 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:57:50.107183 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:50.108868 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:50.123977 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3e8c-9c5c-4122-9036-7531eee01344 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:50.190195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:50.190817 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:50.191095 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:50.191373 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] PATH_INFO: `/v3/users/2bd587c1c2b94aee9ced936892c3c059` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:50.192620 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd587c1c2b94aee9ced936892c3c059)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:50.194594 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:50.196153 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:57:50.245506 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd587c1c2b94aee9ced936892c3c059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.246269 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd587c1c2b94aee9ced936892c3c059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.253798 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd587c1c2b94aee9ced936892c3c059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.263563 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd587c1c2b94aee9ced936892c3c059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.264167 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd587c1c2b94aee9ced936892c3c059'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.282722 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1434/2895] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:57:50 2022] DELETE /identity/v3/users/2bd587c1c2b94aee9ced936892c3c059 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:57:50.461252 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:57:50.462812 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:57:50.476554 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3690c-fe62-419e-b586-0bca7f1d5f94 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:57:50.528656 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:57:50.542802 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:57:50.543453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:57:50.543735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:57:50.544002 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] PATH_INFO: `/v3/projects/613715b645fb404eb228e746ca3fd077` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:57:50.545207 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613715b645fb404eb228e746ca3fd077)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:57:50.547145 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:57:50.593820 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613715b645fb404eb228e746ca3fd077', 'request_id': 'req-396dd38f-6368-4868-a87e-2aef3ba72b07'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.601961 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613715b645fb404eb228e746ca3fd077', 'request_id': 'req-396dd38f-6368-4868-a87e-2aef3ba72b07'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.607857 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613715b645fb404eb228e746ca3fd077', 'request_id': 'req-396dd38f-6368-4868-a87e-2aef3ba72b07'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.619235 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] The token cache is being invalidate because project 613715b645fb404eb228e746ca3fd077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:57:50.619519 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.620248 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:57:50.622050 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1462/2896] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:57:50 2022] DELETE /identity/v3/projects/613715b645fb404eb228e746ca3fd077 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:58:00.278472 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:58:00.615950 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:21.741159 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:21.742824 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:21.805998 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02421c-7fd4-43ad-b8fb-4490553732d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:21.883929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:21.886088 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:21.887323 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:21.888021 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:21.888353 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:21.888616 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:21.890423 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:21.900022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dd38f-6368-4868-a87e-2aef3ba72b07 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:21.901214 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:21.902763 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:21.904233 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:21.904399 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:21.904653 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:21.904929 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:21.907731 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:21.915570 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1435/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:59:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:59:21.993542 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:21.994617 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:22.012999 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1463/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:59:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 22:59:23.226551 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:23.228151 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:23.239514 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159137da-255a-448e-856a-7c6f0ccf1f7d service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:23.243639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:23.244309 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:23.244633 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:23.244953 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:23.246867 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:23.263315 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:23.264914 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:23.279031 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1436/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:59:24.207923 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:24.209709 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:24.225022 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717979-6858-4be3-b04c-d1a104a6e4cf service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:24.232666 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74c1e8e8d3643db9230f448af705d3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:24.233391 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:24.233898 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:24.234382 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:24.236317 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:24.250181 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:24.251817 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:24.263684 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1464/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:59:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:59:33.272951 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=52 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:34.259005 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=40 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:34.383790 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:34.393290 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da18d-9275-4735-ba66-f6e4bcc7366b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:34.397032 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c4de72ad26844d3a0dd6d8b26159bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:34.397680 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:34.398195 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:34.398473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:34.400270 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:34.467519 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:34.468994 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:34.480033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1437/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 22:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 22:59:40.962767 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:40.965534 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:40.983446 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168b35-5e1e-4669-84f2-e1ba8c9b6b1c service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:40.988051 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:40.988787 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:40.989122 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:40.989680 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] PATH_INFO: `/v3/users/4bba292314fd415f94fb537237d9d898` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:40.990873 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bba292314fd415f94fb537237d9d898)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:40.992786 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:41.047280 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bba292314fd415f94fb537237d9d898'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.056078 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bba292314fd415f94fb537237d9d898'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.056987 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bba292314fd415f94fb537237d9d898'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.057771 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bba292314fd415f94fb537237d9d898'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.071568 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bba292314fd415f94fb537237d9d898'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.086420 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1465/2902] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:59:40 2022] DELETE /identity/v3/users/4bba292314fd415f94fb537237d9d898 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:59:41.266033 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:41.276646 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b614ba3-c874-4d60-8552-bf4d903a8b31 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:41.338347 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:41.339356 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:41.339736 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:41.340130 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] PATH_INFO: `/v3/projects/8b83b190c95f4ed8901f10c9fc67b365` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:41.341984 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b83b190c95f4ed8901f10c9fc67b365)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:41.344872 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:41.399402 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b83b190c95f4ed8901f10c9fc67b365', 'request_id': 'req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.408576 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b83b190c95f4ed8901f10c9fc67b365', 'request_id': 'req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.414080 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b83b190c95f4ed8901f10c9fc67b365', 'request_id': 'req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.425733 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] The token cache is being invalidate because project 8b83b190c95f4ed8901f10c9fc67b365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:59:41.426205 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.427184 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.428833 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1438/2903] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:59:41 2022] DELETE /identity/v3/projects/8b83b190c95f4ed8901f10c9fc67b365 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:59:41.434608 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:41.436825 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:41.453818 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16a401-c7d4-4872-a34a-2d6cee11a746 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:41.515939 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:41.516532 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:41.516808 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:41.517079 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] PATH_INFO: `/v3/users/f555d2973b974f388dab1791b19715a7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:41.518309 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f555d2973b974f388dab1791b19715a7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:41.520354 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:41.570042 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f555d2973b974f388dab1791b19715a7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.578428 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f555d2973b974f388dab1791b19715a7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.579025 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f555d2973b974f388dab1791b19715a7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.579662 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f555d2973b974f388dab1791b19715a7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.591854 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f555d2973b974f388dab1791b19715a7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.593824 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:41.607157 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1466/2904] 10.222.0.47 () {58 vars in 1224 bytes} [Mon Aug 1 22:59:41 2022] DELETE /identity/v3/users/f555d2973b974f388dab1791b19715a7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:59:41.784783 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 22:59:41.786363 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 22:59:41.799440 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa908a85-b056-4e2f-94ae-efd6d5e5adf2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 22:59:41.823731 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:41.864956 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47de1e2ad35749d7a331fc838348d562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e92bd71b1dab4dce929247e99aba6067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 22:59:41.865605 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 22:59:41.865908 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 22:59:41.866178 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] PATH_INFO: `/v3/projects/06ab7796b1b241628b1b854be252b709` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 22:59:41.867468 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ab7796b1b241628b1b854be252b709)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 22:59:41.873656 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 22:59:41.940561 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ab7796b1b241628b1b854be252b709', 'request_id': 'req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.950565 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ab7796b1b241628b1b854be252b709', 'request_id': 'req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.956473 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ab7796b1b241628b1b854be252b709', 'request_id': 'req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.967639 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] The token cache is being invalidate because project 06ab7796b1b241628b1b854be252b709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 22:59:41.967923 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.968552 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bd42fe96-0fe9-4bef-8f0e-7631dff1f05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 22:59:41.970148 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1439/2905] 10.222.0.47 () {58 vars in 1233 bytes} [Mon Aug 1 22:59:41 2022] DELETE /identity/v3/projects/06ab7796b1b241628b1b854be252b709 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 22:59:51.608359 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 22:59:51.964917 dv-cinder-851640-1-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 23:02:21.984962 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 23:02:21.986675 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 23:02:22.134663 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa5442-605e-43e6-aebc-8eb809de5be7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 23:02:22.195835 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30e6ac9cc1264e109a68bc20fd975623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '668cc085aa8f430e93314a3a57211a59', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 23:02:22.196453 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 23:02:22.196756 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 23:02:22.197201 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 23:02:22.199068 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 23:02:22.211237 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 23:02:22.212735 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bfc43-8f89-4535-a63b-1e8a8b567391 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 23:02:22.223681 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1467/2906] 10.222.0.47 () {60 vars in 1365 bytes} [Mon Aug 1 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 23:02:32.218797 dv-cinder-851640-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}